[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:21 dist-test-slave-l4q5 krb5kdc[1561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:21 dist-test-slave-l4q5 krb5kdc[1561](info): set up 2 sockets
Sep 23 22:12:21 dist-test-slave-l4q5 krb5kdc[1561](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 22:12:22.385318 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:22.504132 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36815
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:22.585660 1574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:22.585876 1574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:22.585898 1574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:22.587287 1574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:22.587347 1574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:22.587363 1574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:22.587373 1574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:22.587384 1574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:22.589082 1574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:22.589321 1574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:22.589563 1574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:22.594921 1580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.594990 1582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.594923 1579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.594956 1574 server_base.cc:1034] running on GCE node
I20260923 22:12:22.595381 1574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:22.595717 1574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:22.596886 1574 hybrid_clock.cc:648] HybridClock initialized: now 1790201542596862 us; error 41 us; skew 500 ppm
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:22.599809 1574 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:22.600404 1574 webserver.cc:460] Webserver started at http://127.1.132.190:46571/ using document root <none> and password file <none>
I20260923 22:12:22.600642 1574 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:22.600692 1574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:22.600802 1574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:22.601895 1574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "3d206a09dda440a7b65d94cb975f760f"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "455b52177e3e97ec7bbdc4651d9cdb80"
server_key_iv: "11f3323d3a6bf7fcf715389ea5e7b30c"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.602236 1574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "3d206a09dda440a7b65d94cb975f760f"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "455b52177e3e97ec7bbdc4651d9cdb80"
server_key_iv: "11f3323d3a6bf7fcf715389ea5e7b30c"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.603694 1574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:22.604498 1589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.604676 1574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:22.604761 1574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "3d206a09dda440a7b65d94cb975f760f"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "455b52177e3e97ec7bbdc4651d9cdb80"
server_key_iv: "11f3323d3a6bf7fcf715389ea5e7b30c"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.604835 1574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:22.624439 1574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:22.626703 1574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:22.626843 1574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:22.630942 1574 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36815
I20260923 22:12:22.630986 1641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36815 every 8 connection(s)
I20260923 22:12:22.631356 1574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:22.632257 1642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:22.634570 1642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: Bootstrap starting.
I20260923 22:12:22.635147 1642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:22.635403 1642 log.cc:824] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:22.636238 1642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: No bootstrap required, opened a new log
I20260923 22:12:22.637848 1642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } }
I20260923 22:12:22.638028 1642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:22.638062 1642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d206a09dda440a7b65d94cb975f760f, State: Initialized, Role: FOLLOWER
I20260923 22:12:22.638208 1642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } }
I20260923 22:12:22.638286 1642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:22.638326 1642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:22.638375 1642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:22.638960 1642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } }
I20260923 22:12:22.639057 1642 leader_election.cc:302] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d206a09dda440a7b65d94cb975f760f; no voters:
I20260923 22:12:22.639204 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 1574
I20260923 22:12:22.639238 1642 leader_election.cc:288] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:22.639272 1645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:22.639304 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:22.639398 1645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [term 1 LEADER]: Becoming Leader. State: Replica: 3d206a09dda440a7b65d94cb975f760f, State: Running, Role: LEADER
I20260923 22:12:22.639477 1645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } }
I20260923 22:12:22.639542 1642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:22.639796 1645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d206a09dda440a7b65d94cb975f760f. Latest consensus state: current_term: 1 leader_uuid: "3d206a09dda440a7b65d94cb975f760f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } } }
I20260923 22:12:22.639853 1645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:22.639967 1646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d206a09dda440a7b65d94cb975f760f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d206a09dda440a7b65d94cb975f760f" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36815 } } }
I20260923 22:12:22.640043 1646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:22.640244 1554 external_mini_cluster.cc:1428] Setting key 6f71783d5414bdc65197ee4f37b6f1aa
W20260923 22:12:22.641280 1661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:22.641347 1661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:22.641408 1649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:22.641558 1649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:22.643060 1649 catalog_manager.cc:1347] Generated new cluster ID: 25d8aa02099a4023ac523f9d34659a3a
I20260923 22:12:22.643114 1649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:22.662087 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.641731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.1.132.190:36815)
0923 22:12:22.641909 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:22.641918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:22.642142 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:12:22.644633 (+ 2491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:22.644641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:22.644704 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:22.644863 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:22.646797 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.648750 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.650338 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.651016 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.651137 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.656312 (+ 5175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:22.656318 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:22.656319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:22.656334 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:22.658498 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.659124 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.659128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.659128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.659255 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.659671 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.659672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.659673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.660740 (+ 1067us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:22.660927 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:22.661284 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 22:12:22.661456 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260923 22:12:22.683382 1649 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:22.683976 1649 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:22.687115 1649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d206a09dda440a7b65d94cb975f760f: Generated new TSK 0
I20260923 22:12:22.687297 1649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:22.716101 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:22.798260 1668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:22.798489 1668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:22.798516 1668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:22.799708 1668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:22.799762 1668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:22.799795 1668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:22.801393 1668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:22.801642 1668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:22.801877 1668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:22.806933 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.806938 1676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.807071 1668 server_base.cc:1034] running on GCE node
W20260923 22:12:22.807197 1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.807406 1668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:22.807669 1668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:22.808848 1668 hybrid_clock.cc:648] HybridClock initialized: now 1790201542808833 us; error 29 us; skew 500 ppm
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:22.811324 1668 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:22.811872 1668 webserver.cc:460] Webserver started at http://127.1.132.129:40261/ using document root <none> and password file <none>
I20260923 22:12:22.812100 1668 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:22.812150 1668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:22.812309 1668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:22.813357 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "0e8f73c0f90c4d4381fe0203edfae05e"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "237f10441f91dedd935178b1539babff"
server_key_iv: "24eed3d34f0bf474860ddd296a305f06"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.813704 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "0e8f73c0f90c4d4381fe0203edfae05e"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "237f10441f91dedd935178b1539babff"
server_key_iv: "24eed3d34f0bf474860ddd296a305f06"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.815006 1668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:22.815799 1683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.815953 1668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:22.816038 1668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "0e8f73c0f90c4d4381fe0203edfae05e"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "237f10441f91dedd935178b1539babff"
server_key_iv: "24eed3d34f0bf474860ddd296a305f06"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.816102 1668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:22.839099 1668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:22.841847 1668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:22.841998 1668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:22.842252 1668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:22.842573 1668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:22.842608 1668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.842636 1668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:22.842660 1668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.849063 1668 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:45493
I20260923 22:12:22.849112 1796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:45493 every 8 connection(s)
I20260923 22:12:22.849464 1668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:22.851356 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 1668
I20260923 22:12:22.851429 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:22.851573 1554 external_mini_cluster.cc:1428] Setting key 09553a6e35bbf4f7b97b529b79b181d5
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:12:22.859050 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.850234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58107 (local address 127.1.132.190:36815)
0923 22:12:22.850278 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:22.850280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:22.850526 (+ 246us) server_negotiation.cc:407] Connection header received
0923 22:12:22.851016 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:22.851017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:22.851028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:22.851078 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:22.852079 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.852642 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.853635 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.853853 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.853911 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.856222 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:22.856224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:22.856224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:22.856231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:22.857538 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.857954 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.857955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.857956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.858013 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:22.858232 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:22.858233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:22.858234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:22.858350 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:22.858406 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:22.858682 (+ 276us) server_negotiation.cc:299] Negotiation successful
0923 22:12:22.858926 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:22.859237 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:22.850212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36815 (local address 127.1.132.129:58107)
0923 22:12:22.850401 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:22.850440 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:22.850926 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:22.851137 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:22.851140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:22.851767 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:22.851981 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:22.851988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.852731 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.852733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:22.853554 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:22.853556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:22.853927 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:22.853927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:22.854060 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:22.854578 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:22.854588 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:22.856098 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:22.857629 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:22.857637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:22.857640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:22.857877 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:22.858069 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:22.858070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:22.858070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:22.858177 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:22.858405 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:22.858412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:22.858560 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 22:12:22.858651 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:22.858983 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260923 22:12:22.859736 1797 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36815
I20260923 22:12:22.859836 1797 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:22.860069 1797 heartbeater.cc:511] Master 127.1.132.190:36815 requested a full tablet report, sending...
I20260923 22:12:22.860623 1606 ts_manager.cc:194] Registered new tserver with Master: 0e8f73c0f90c4d4381fe0203edfae05e (127.1.132.129:45493)
I20260923 22:12:22.861268 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:58107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:22.880229 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:22.963755 1802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:22.963974 1802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:22.963999 1802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:22.965149 1802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:22.965188 1802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:22.965214 1802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:22.966622 1802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:22.966823 1802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:22.967005 1802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:22.972504 1808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:22.972505 1810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.972524 1802 server_base.cc:1034] running on GCE node
W20260923 22:12:22.972512 1807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:22.972961 1802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:22.973178 1802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:22.974313 1802 hybrid_clock.cc:648] HybridClock initialized: now 1790201542974293 us; error 48 us; skew 500 ppm
Sep 23 22:12:22 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:22.976894 1802 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:22.977403 1802 webserver.cc:460] Webserver started at http://127.1.132.130:34973/ using document root <none> and password file <none>
I20260923 22:12:22.977632 1802 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:22.977782 1802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:22.977975 1802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:22.979089 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "7ffab4f7ca684e699ee7fcc6afcb6edb"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "3be8b4da50ac96d5c07ced0a839887d6"
server_key_iv: "25ab76df11548b6c35289b6868db5100"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.979457 1802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "7ffab4f7ca684e699ee7fcc6afcb6edb"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "3be8b4da50ac96d5c07ced0a839887d6"
server_key_iv: "25ab76df11548b6c35289b6868db5100"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.980929 1802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:22.981782 1817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:22.981940 1802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:22.982059 1802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "7ffab4f7ca684e699ee7fcc6afcb6edb"
format_stamp: "Formatted at 2026-09-23 22:12:22 on dist-test-slave-l4q5"
server_key: "3be8b4da50ac96d5c07ced0a839887d6"
server_key_iv: "25ab76df11548b6c35289b6868db5100"
server_key_version: "encryptionkey@0"
I20260923 22:12:22.982127 1802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:23.022356 1802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:23.024427 1802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:23.024593 1802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:23.024821 1802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:23.025125 1802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:23.025170 1802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.025200 1802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:23.025216 1802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.031837 1802 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:41887
I20260923 22:12:23.031896 1930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:41887 every 8 connection(s)
I20260923 22:12:23.032222 1802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:23.035429 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 1802
I20260923 22:12:23.035538 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:23.035748 1554 external_mini_cluster.cc:1428] Setting key 11c29ef07a86bcffea56c720a9b2adfc
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:23.043433 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.032996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:54801 (local address 127.1.132.190:36815)
0923 22:12:23.033063 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.033065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.033252 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:12:23.033774 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.033776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.033786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.033874 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.035072 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.035632 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.036993 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.037189 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.037253 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.040059 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.040061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.040061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.040070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.041812 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.042264 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.042266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.042266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.042334 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.042679 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.042680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.042681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.042836 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.042908 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.043233 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.043333 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:23.043565 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.032959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36815 (local address 127.1.132.130:54801)
0923 22:12:23.033136 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.033175 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.033665 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.033992 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.033996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.034608 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:23.034907 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.034913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.035774 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.035775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.036886 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.036889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.037278 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.037279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.037433 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.038245 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:23.038254 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:23.039948 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:23.041922 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.041930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.041933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.042186 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.042465 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.042466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.042466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.042603 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.042928 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:23.042935 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:23.043090 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.043191 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.043335 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260923 22:12:23.044534 1931 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36815
I20260923 22:12:23.044642 1931 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:23.044834 1931 heartbeater.cc:511] Master 127.1.132.190:36815 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:12:23.045328 1606 ts_manager.cc:194] Registered new tserver with Master: 7ffab4f7ca684e699ee7fcc6afcb6edb (127.1.132.130:41887)
I20260923 22:12:23.045727 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:54801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:23.068112 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:23.149961 1936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:23.150183 1936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:23.150216 1936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:23.151371 1936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:23.151413 1936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:23.151449 1936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:23.152902 1936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36815
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:23.153158 1936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:23.153367 1936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:23.158267 1942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:23.158286 1941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:23.158589 1936 server_base.cc:1034] running on GCE node
W20260923 22:12:23.158262 1944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:23.158838 1936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:23.159055 1936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:23.160226 1936 hybrid_clock.cc:648] HybridClock initialized: now 1790201543160210 us; error 28 us; skew 500 ppm
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:23.162643 1936 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:23.163106 1936 webserver.cc:460] Webserver started at http://127.1.132.131:44845/ using document root <none> and password file <none>
I20260923 22:12:23.163303 1936 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:23.163348 1936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:23.163461 1936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:23.164557 1936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "70c9b0c073b94fedad93b382dcf24c14"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-l4q5"
server_key: "74fd3a0e4fe5b53a69e3ec0a34f4517c"
server_key_iv: "881866b08d8b1eae62973dd83b9f353a"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.164868 1936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "70c9b0c073b94fedad93b382dcf24c14"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-l4q5"
server_key: "74fd3a0e4fe5b53a69e3ec0a34f4517c"
server_key_iv: "881866b08d8b1eae62973dd83b9f353a"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.166095 1936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:23.166986 1951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.167191 1936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:23.167273 1936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "70c9b0c073b94fedad93b382dcf24c14"
format_stamp: "Formatted at 2026-09-23 22:12:23 on dist-test-slave-l4q5"
server_key: "74fd3a0e4fe5b53a69e3ec0a34f4517c"
server_key_iv: "881866b08d8b1eae62973dd83b9f353a"
server_key_version: "encryptionkey@0"
I20260923 22:12:23.167331 1936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:23.184249 1936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:23.186585 1936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:23.186733 1936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:23.186993 1936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:23.187317 1936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:23.187352 1936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.187381 1936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:23.187407 1936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:23.194525 1936 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:43557
I20260923 22:12:23.194563 2064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:43557 every 8 connection(s)
I20260923 22:12:23.194877 1936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:23.202787 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 1936
I20260923 22:12:23.202862 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:23.203008 1554 external_mini_cluster.cc:1428] Setting key 5ed7102465cf9f1043c9c6201ede7b56
I20260923 22:12:23.205225 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.195646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:57589 (local address 127.1.132.190:36815)
0923 22:12:23.195708 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.195710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.195975 (+ 265us) server_negotiation.cc:407] Connection header received
0923 22:12:23.196645 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.196646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.196662 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.196720 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.197815 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.198334 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.199393 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.199613 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.199671 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.201961 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.201963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.201964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.201971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.203359 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.203869 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.203870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.203871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.204018 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.204384 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.204385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.204386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.204541 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.204611 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.205040 (+ 429us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.205134 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:12:23.205474 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.195610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36815 (local address 127.1.132.131:57589)
0923 22:12:23.195847 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:23.195887 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:23.196519 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:23.196843 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:23.196847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:23.197443 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:23.197701 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.197706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.198431 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.198432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.199294 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:23.199296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.199690 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.199691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:23.199805 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.200359 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:23.200368 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:23.201856 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:23.203459 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.203469 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.203471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.203801 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.204113 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:23.204114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:23.204114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:23.204308 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:23.204622 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:23.204630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:23.204817 (+ 187us) client_negotiation.cc:769] Sending connection context
0923 22:12:23.204940 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:12:23.205142 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":127,"threads_started":1}
I20260923 22:12:23.207718 2065 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36815
I20260923 22:12:23.207839 2065 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:23.208050 2065 heartbeater.cc:511] Master 127.1.132.190:36815 requested a full tablet report, sending...
I20260923 22:12:23.208513 1606 ts_manager.cc:194] Registered new tserver with Master: 70c9b0c073b94fedad93b382dcf24c14 (127.1.132.131:43557)
I20260923 22:12:23.209029 1606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:57589
I20260923 22:12:23.215255 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:23.238673 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.229913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.1.132.129:45493)
0923 22:12:23.230123 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.230128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.230139 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:23.230165 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.230167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.230182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.230246 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.230849 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.231736 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.232682 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.233179 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.233256 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.235873 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.235884 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.235884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.235892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.237616 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.237978 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.237982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.237983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.238049 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.238179 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.238179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.238179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.238311 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.238434 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.238472 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.238564 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
W20260923 22:12:23.239130 1711 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56588
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[1561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:23.262249 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:23.254869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.1.132.129:45493)
0923 22:12:23.254943 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:23.254945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:23.254955 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:23.254980 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:23.255001 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:23.255013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:23.255068 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:23.255724 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.256298 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.257204 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:23.257419 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:23.257585 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:23.259884 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:23.259886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:23.259886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:23.259896 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:23.261225 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.261550 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.261551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.261551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.261598 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:23.261767 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:23.261768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:23.261769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:23.261888 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:23.261995 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:23.262104 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:12:23.262179 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:12:23.262753 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 1668
I20260923 22:12:23.267153 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 1802
I20260923 22:12:23.272253 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 1936
I20260923 22:12:23.276149 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 1574
2026-09-23T22:12:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1862 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[2079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[2079](info): set up 2 sockets
Sep 23 22:12:23 dist-test-slave-l4q5 krb5kdc[2079](info): commencing operation
krb5kdc: starting...
W20260923 22:12:24.243548 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:24.359071 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35661
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.443003 2092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.443217 2092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.443252 2092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.444475 2092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:24.444518 2092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.444537 2092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.444552 2092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:24.444567 2092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:24.446053 2092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:24.446301 2092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.446516 2092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:24.451493 2100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.451508 2097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.451594 2092 server_base.cc:1034] running on GCE node
W20260923 22:12:24.451519 2098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.451977 2092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.452299 2092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.453457 2092 hybrid_clock.cc:648] HybridClock initialized: now 1790201544453437 us; error 31 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.455977 2092 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:24.456467 2092 webserver.cc:460] Webserver started at http://127.1.132.190:36765/ using document root <none> and password file <none>
I20260923 22:12:24.456691 2092 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.456737 2092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.456843 2092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.457909 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "f7331e81252243618d234b033926e19a"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "bc63d5e0ab3e8be43fbb7b43c402365c"
server_key_iv: "71c26345089b20293b388aef9f88091d"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.458236 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "f7331e81252243618d234b033926e19a"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "bc63d5e0ab3e8be43fbb7b43c402365c"
server_key_iv: "71c26345089b20293b388aef9f88091d"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.459807 2092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:24.460801 2107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.461138 2092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:24.461234 2092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "f7331e81252243618d234b033926e19a"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "bc63d5e0ab3e8be43fbb7b43c402365c"
server_key_iv: "71c26345089b20293b388aef9f88091d"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.461324 2092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.469643 2092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.471745 2092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.471879 2092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.475795 2092 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35661
I20260923 22:12:24.475835 2159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35661 every 8 connection(s)
I20260923 22:12:24.476217 2092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:24.477087 2160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:24.479409 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a: Bootstrap starting.
I20260923 22:12:24.480000 2160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:24.480295 2160 log.cc:824] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:24.481072 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a: No bootstrap required, opened a new log
I20260923 22:12:24.482764 2160 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } }
I20260923 22:12:24.482928 2160 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:24.482961 2160 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7331e81252243618d234b033926e19a, State: Initialized, Role: FOLLOWER
I20260923 22:12:24.483105 2160 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } }
I20260923 22:12:24.483178 2160 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:24.483203 2160 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:24.483253 2160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:24.483815 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2092
I20260923 22:12:24.483906 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:24.483935 2160 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } }
I20260923 22:12:24.484067 1554 external_mini_cluster.cc:1428] Setting key 9649ffca8114a1ce15915169ee281c76
I20260923 22:12:24.484054 2160 leader_election.cc:302] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7331e81252243618d234b033926e19a; no voters:
I20260923 22:12:24.484274 2160 leader_election.cc:288] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:24.484509 2163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:24.484665 2163 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [term 1 LEADER]: Becoming Leader. State: Replica: f7331e81252243618d234b033926e19a, State: Running, Role: LEADER
I20260923 22:12:24.484761 2163 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } }
I20260923 22:12:24.484885 2160 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:24.485231 2163 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7331e81252243618d234b033926e19a. Latest consensus state: current_term: 1 leader_uuid: "f7331e81252243618d234b033926e19a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } } }
I20260923 22:12:24.485240 2166 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7331e81252243618d234b033926e19a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7331e81252243618d234b033926e19a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35661 } } }
I20260923 22:12:24.485394 2166 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:24.485333 2163 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:24.486037 2169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:24.488174 2169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:24.489988 2169 catalog_manager.cc:1347] Generated new cluster ID: 853b126d81da4f29be840aa244c084c3
I20260923 22:12:24.490052 2169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:24.495333 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.484786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34274 (local address 127.1.132.190:35661)
0923 22:12:24.485263 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.485272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.485309 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:12:24.485630 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.485638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.485705 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.485858 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.486537 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.487927 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.488899 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.489432 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.489567 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.492085 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.492091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.492091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.492106 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.493796 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.494144 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.494148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.494148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.494256 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.494437 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.494438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.494438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.494707 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.494906 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.494987 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.495133 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":83,"threads_started":1}
I20260923 22:12:24.509328 2169 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:24.509939 2169 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:24.515641 2169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7331e81252243618d234b033926e19a: Generated new TSK 0
I20260923 22:12:24.515847 2169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:24.549345 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.662834 2186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.663096 2186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.663125 2186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.664418 2186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.664469 2186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.664497 2186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:24.666119 2186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:24.666328 2186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.666517 2186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:24.671448 2192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.671476 2186 server_base.cc:1034] running on GCE node
W20260923 22:12:24.671494 2194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.671448 2191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.671964 2186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.672240 2186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.673398 2186 hybrid_clock.cc:648] HybridClock initialized: now 1790201544673387 us; error 30 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.675925 2186 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:24.676489 2186 webserver.cc:460] Webserver started at http://127.1.132.129:42439/ using document root <none> and password file <none>
I20260923 22:12:24.676694 2186 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.676739 2186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.676851 2186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.677938 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "118ba28c875b43c4993c0cf750e7f60b"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "529c74cc972d4c3977c6ab6dfcf17223"
server_key_iv: "8b9ecfc1e0011494904d07903041e1cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.678270 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "118ba28c875b43c4993c0cf750e7f60b"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "529c74cc972d4c3977c6ab6dfcf17223"
server_key_iv: "8b9ecfc1e0011494904d07903041e1cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.679595 2186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:24.680370 2201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.680552 2186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:24.680640 2186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "118ba28c875b43c4993c0cf750e7f60b"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "529c74cc972d4c3977c6ab6dfcf17223"
server_key_iv: "8b9ecfc1e0011494904d07903041e1cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.680699 2186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.691236 2186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.693372 2186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.693499 2186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.693691 2186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:24.693953 2186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:24.693985 2186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.694011 2186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:24.694027 2186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.699276 2186 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:40513
I20260923 22:12:24.699332 2314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:40513 every 8 connection(s)
I20260923 22:12:24.699656 2186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:24.704917 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2186
I20260923 22:12:24.705045 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:24.705207 1554 external_mini_cluster.cc:1428] Setting key 78b65ee6bd0766135dec8147d6db5809
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:24.709545 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.700391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:46817 (local address 127.1.132.190:35661)
0923 22:12:24.700473 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.700475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.700671 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:12:24.701620 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.701621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.701633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.701686 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.702321 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.702819 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.703821 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.704028 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.704083 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.706569 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.706570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.706570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.706577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.707905 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.708385 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.708386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.708387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.708470 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.708815 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.708816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.708817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.708946 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.709012 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.709400 (+ 388us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.709477 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:12:24.709744 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.700358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35661 (local address 127.1.132.129:46817)
0923 22:12:24.700541 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:24.700586 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:24.701507 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:24.701764 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:24.701768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:24.702028 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:24.702226 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.702230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.702907 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.702908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.703734 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.703737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.704100 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.704101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.704259 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.704750 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:24.704761 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:24.706469 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:24.707981 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.707988 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.707991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.708304 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.708533 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.708534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.708534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.708663 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.709017 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:24.709023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:24.709205 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 22:12:24.709320 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 22:12:24.709505 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260923 22:12:24.710302 2315 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35661
I20260923 22:12:24.710410 2315 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:24.710623 2315 heartbeater.cc:511] Master 127.1.132.190:35661 requested a full tablet report, sending...
I20260923 22:12:24.711145 2124 ts_manager.cc:194] Registered new tserver with Master: 118ba28c875b43c4993c0cf750e7f60b (127.1.132.129:40513)
I20260923 22:12:24.711736 2124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:46817
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:24.730394 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.811216 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.811430 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.811455 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.812695 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.812742 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.812770 2320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:24.814327 2320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:24.814581 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.814813 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:24.820091 2320 server_base.cc:1034] running on GCE node
W20260923 22:12:24.820052 2325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.820052 2328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.820052 2326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.820477 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.820719 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.821880 2320 hybrid_clock.cc:648] HybridClock initialized: now 1790201544821866 us; error 28 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.824414 2320 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:24.824910 2320 webserver.cc:460] Webserver started at http://127.1.132.130:34469/ using document root <none> and password file <none>
I20260923 22:12:24.825148 2320 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.825197 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.825328 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.826355 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "ba251b4d64564c22bb2ba647e3fa87cd"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "f93db75a09001eda6c235d47d020d4ac"
server_key_iv: "5e44122b8de9bdc9f3a341ac70e48c3e"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.826686 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "ba251b4d64564c22bb2ba647e3fa87cd"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "f93db75a09001eda6c235d47d020d4ac"
server_key_iv: "5e44122b8de9bdc9f3a341ac70e48c3e"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.827970 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:24.828778 2335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.828923 2320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:24.829018 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "ba251b4d64564c22bb2ba647e3fa87cd"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "f93db75a09001eda6c235d47d020d4ac"
server_key_iv: "5e44122b8de9bdc9f3a341ac70e48c3e"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.829084 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.843268 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.845407 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.845566 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.845920 2320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:24.846274 2320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:24.846312 2320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.846347 2320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:24.846364 2320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.851984 2320 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:37693
I20260923 22:12:24.852036 2448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:37693 every 8 connection(s)
I20260923 22:12:24.852403 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:24.855048 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2320
I20260923 22:12:24.855125 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:24.855262 1554 external_mini_cluster.cc:1428] Setting key d3179d70232a34f04609776dfa0afe86
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:24.862270 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.853121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:35873 (local address 127.1.132.190:35661)
0923 22:12:24.853166 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.853168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.853338 (+ 170us) server_negotiation.cc:407] Connection header received
0923 22:12:24.854237 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.854237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.854247 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.854308 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.854875 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.855481 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.856969 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.857193 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.857280 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.859368 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:24.859370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:24.859370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:24.859379 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:24.860830 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.861327 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.861328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.861329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.861385 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:24.861651 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:24.861652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:24.861652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:24.861765 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:24.861811 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:24.862140 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 22:12:24.862204 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:24.862538 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.853069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35661 (local address 127.1.132.130:35873)
0923 22:12:24.853214 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:24.853258 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:24.854153 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:24.854364 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:24.854367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:24.854619 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:24.854812 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.854816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.855558 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.855559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.856843 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.856846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.857309 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.857309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.857445 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.857936 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:24.857945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:24.859259 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:24.860917 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.860925 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.860928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.861244 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.861448 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:24.861449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:24.861449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:24.861594 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:24.861819 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:24.861824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:24.862006 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 22:12:24.862122 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:12:24.862296 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:12:24.863073 2449 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35661
I20260923 22:12:24.863178 2449 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:24.863359 2449 heartbeater.cc:511] Master 127.1.132.190:35661 requested a full tablet report, sending...
I20260923 22:12:24.863754 2124 ts_manager.cc:194] Registered new tserver with Master: ba251b4d64564c22bb2ba647e3fa87cd (127.1.132.130:37693)
I20260923 22:12:24.864230 2124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:35873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:24.881137 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:24.959721 2454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:24.959930 2454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:24.959951 2454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:24.961125 2454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:24.961169 2454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:24.961194 2454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:24.962622 2454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:24.962808 2454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:24.962996 2454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:24.968055 2454 server_base.cc:1034] running on GCE node
W20260923 22:12:24.968057 2460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.967995 2459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:24.968011 2462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:24.968405 2454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:24.968675 2454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:24.969813 2454 hybrid_clock.cc:648] HybridClock initialized: now 1790201544969796 us; error 31 us; skew 500 ppm
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:24.972203 2454 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:24.972697 2454 webserver.cc:460] Webserver started at http://127.1.132.131:45679/ using document root <none> and password file <none>
I20260923 22:12:24.972944 2454 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:24.972991 2454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:24.973086 2454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:24.974080 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "605278fa3ca6411ea8a05d1f6ec711ea"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "107cf7562096ef1cdd6f69d9d19c6661"
server_key_iv: "ccb43b6fdd482dd4508441c74902d0ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.974360 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "605278fa3ca6411ea8a05d1f6ec711ea"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "107cf7562096ef1cdd6f69d9d19c6661"
server_key_iv: "ccb43b6fdd482dd4508441c74902d0ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.975641 2454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:24.976290 2469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.976488 2454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.976567 2454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "605278fa3ca6411ea8a05d1f6ec711ea"
format_stamp: "Formatted at 2026-09-23 22:12:24 on dist-test-slave-l4q5"
server_key: "107cf7562096ef1cdd6f69d9d19c6661"
server_key_iv: "ccb43b6fdd482dd4508441c74902d0ae"
server_key_version: "encryptionkey@0"
I20260923 22:12:24.976629 2454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:24.984663 2454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:24.987028 2454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:24.987150 2454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:24.987375 2454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:24.987671 2454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:24.987705 2454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.987726 2454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:24.987737 2454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:24.993121 2454 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:36497
I20260923 22:12:24.993178 2582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:36497 every 8 connection(s)
I20260923 22:12:24.993498 2454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:24.995762 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2454
I20260923 22:12:24.995849 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:24.995993 1554 external_mini_cluster.cc:1428] Setting key 3a56dd7c0abcc536f74543f3fbb64c4b
Sep 23 22:12:24 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:25.003714 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.994324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:47907 (local address 127.1.132.190:35661)
0923 22:12:24.994399 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:24.994402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:24.994540 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:12:24.995503 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:24.995505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:24.995517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:24.995570 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:24.996321 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.997081 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.998165 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.998369 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.998426 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.001049 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.001051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.001051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.001059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.002378 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.002834 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.002835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.002835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.002889 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.003106 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.003107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.003107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.003226 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.003282 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.003540 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.003620 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:12:25.003899 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:24.994266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35661 (local address 127.1.132.131:47907)
0923 22:12:24.994413 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:24.994459 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:24.995364 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:24.995659 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:24.995662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:24.995957 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:24.996236 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.996244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.997161 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.997163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.998071 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:24.998074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:24.998555 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:24.998559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:24.998711 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:24.999395 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:24.999406 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:25.000943 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:25.002517 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.002526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.002529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.002763 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.002948 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:25.002948 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:25.002949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:25.003052 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:25.003278 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:25.003284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:25.003419 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.003511 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.003671 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260923 22:12:25.004397 2583 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35661
I20260923 22:12:25.004490 2583 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:25.004657 2583 heartbeater.cc:511] Master 127.1.132.190:35661 requested a full tablet report, sending...
I20260923 22:12:25.005055 2124 ts_manager.cc:194] Registered new tserver with Master: 605278fa3ca6411ea8a05d1f6ec711ea (127.1.132.131:36497)
I20260923 22:12:25.005460 2124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:47907
I20260923 22:12:25.007892 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:25.014844 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.009355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.1.132.190:35661)
0923 22:12:25.009418 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.009420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.009509 (+ 89us) server_negotiation.cc:407] Connection header received
0923 22:12:25.009550 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.009551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.009559 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.009606 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.010088 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.010559 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.011299 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.011505 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.011567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.012700 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.012702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.012702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.012708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.014010 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.014267 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.014268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.014268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.014312 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.014437 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.014437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.014438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.014577 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.014645 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.014725 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.014779 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:25.016973 2124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:25.018415 2124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:25.024392 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:37693 (local address 127.0.0.1:49770)
0923 22:12:25.020904 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.020936 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.021020 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.021216 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.021217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.021220 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.021307 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.021309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.022284 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.022285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.023219 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.023221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.024080 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.024081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.024161 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.024175 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.024265 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.024311 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":57,"thread_start_us":46,"threads_started":1}
I20260923 22:12:25.024392 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.1.132.130:37693)
0923 22:12:25.020848 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.020853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.021093 (+ 240us) server_negotiation.cc:407] Connection header received
0923 22:12:25.021134 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.021136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.021148 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.021209 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.021363 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.022187 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.023297 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.024002 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.024071 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.024128 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.024277 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.024315 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260923 22:12:25.024740 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:36497 (local address 127.0.0.1:57780)
0923 22:12:25.020906 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.020948 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.021007 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.021212 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.021216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.021218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.021306 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.021309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.022265 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.022265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.023582 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.023584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.024500 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.024501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.024574 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.024578 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.024627 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.024672 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":53,"thread_start_us":70,"threads_started":1}
I20260923 22:12:25.024941 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.1.132.131:36497)
0923 22:12:25.021072 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.021077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.021085 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:25.021121 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.021123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.021137 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.021202 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.021387 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.022174 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.023654 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.024345 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.024404 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.024464 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.024829 (+ 365us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.024868 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":89,"threads_started":1}
I20260923 22:12:25.025312 2383 tablet_service.cc:1467] Processing CreateTablet for tablet 7a96961a20e64101a8996f4564f457f4 (DEFAULT_TABLE table=test-table [id=c536ea252b19477c9b36e52bcad05a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.025642 2383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a96961a20e64101a8996f4564f457f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.025714 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:40513 (local address 127.0.0.1:60664)
0923 22:12:25.021298 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.021307 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.021358 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.022443 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.022444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.022446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.022483 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.022485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.023680 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.023681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.024739 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.024742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.025484 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.025486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.025553 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.025558 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.025616 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.025653 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":62,"threads_started":1}
I20260923 22:12:25.025748 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.020881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.1.132.129:40513)
0923 22:12:25.021047 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.021051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.022294 (+ 1243us) server_negotiation.cc:407] Connection header received
0923 22:12:25.022339 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.022341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.022357 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.022432 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.022542 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.023582 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.024820 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.025404 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.025471 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.025522 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.025631 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.025671 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260923 22:12:25.026268 2249 tablet_service.cc:1467] Processing CreateTablet for tablet 7a96961a20e64101a8996f4564f457f4 (DEFAULT_TABLE table=test-table [id=c536ea252b19477c9b36e52bcad05a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.026531 2249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a96961a20e64101a8996f4564f457f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.028263 2600 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Bootstrap starting.
I20260923 22:12:25.028759 2517 tablet_service.cc:1467] Processing CreateTablet for tablet 7a96961a20e64101a8996f4564f457f4 (DEFAULT_TABLE table=test-table [id=c536ea252b19477c9b36e52bcad05a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:25.028901 2600 tablet_bootstrap.cc:654] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.029059 2601 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: Bootstrap starting.
I20260923 22:12:25.029091 2517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a96961a20e64101a8996f4564f457f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:25.029397 2600 log.cc:824] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.029650 2601 tablet_bootstrap.cc:654] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.030104 2601 log.cc:824] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.030232 2600 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: No bootstrap required, opened a new log
I20260923 22:12:25.030330 2600 ts_tablet_manager.cc:1397] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:25.030681 2601 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: No bootstrap required, opened a new log
I20260923 22:12:25.030737 2601 ts_tablet_manager.cc:1397] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:25.032454 2604 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: Bootstrap starting.
I20260923 22:12:25.032573 2600 raft_consensus.cc:348] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.032994 2600 raft_consensus.cc:374] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.033049 2600 raft_consensus.cc:729] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba251b4d64564c22bb2ba647e3fa87cd, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.033032 2601 raft_consensus.cc:348] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.033203 2601 raft_consensus.cc:374] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.033242 2601 raft_consensus.cc:729] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118ba28c875b43c4993c0cf750e7f60b, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.033200 2600 consensus_queue.cc:260] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.033285 2604 tablet_bootstrap.cc:654] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:25.033375 2601 consensus_queue.cc:260] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.033483 2449 heartbeater.cc:503] Master 127.1.132.190:35661 was elected leader, sending a full tablet report...
I20260923 22:12:25.033643 2601 ts_tablet_manager.cc:1428] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:25.033744 2315 heartbeater.cc:503] Master 127.1.132.190:35661 was elected leader, sending a full tablet report...
I20260923 22:12:25.033844 2604 log.cc:824] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:25.033941 2600 ts_tablet_manager.cc:1428] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:12:25.034590 2604 tablet_bootstrap.cc:492] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: No bootstrap required, opened a new log
I20260923 22:12:25.034663 2604 ts_tablet_manager.cc:1397] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:25.035989 2604 raft_consensus.cc:348] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.036213 2604 raft_consensus.cc:374] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:25.036253 2604 raft_consensus.cc:729] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605278fa3ca6411ea8a05d1f6ec711ea, State: Initialized, Role: FOLLOWER
I20260923 22:12:25.036368 2604 consensus_queue.cc:260] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.036614 2604 ts_tablet_manager.cc:1428] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:25.036639 2583 heartbeater.cc:503] Master 127.1.132.190:35661 was elected leader, sending a full tablet report...
W20260923 22:12:25.102944 2450 tablet.cc:2369] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:25.105855 2608 raft_consensus.cc:482] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:25.106019 2608 raft_consensus.cc:504] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.106354 2608 leader_election.cc:288] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba251b4d64564c22bb2ba647e3fa87cd (127.1.132.130:37693), 118ba28c875b43c4993c0cf750e7f60b (127.1.132.129:40513)
I20260923 22:12:25.109220 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.106661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59987 (local address 127.1.132.129:40513)
0923 22:12:25.106707 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.106709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.106857 (+ 148us) server_negotiation.cc:407] Connection header received
0923 22:12:25.106886 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.106887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.106899 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.106954 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.107089 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.107551 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108545 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.108906 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108974 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.108981 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.109108 (+ 127us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.109159 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:25.109215 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.106630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:47817 (local address 127.1.132.130:37693)
0923 22:12:25.106686 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.106689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.106698 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:25.106725 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.106726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.106737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.106792 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:25.106965 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.107469 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108491 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.108889 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108956 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.108965 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.109116 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.109151 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:25.109215 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.106559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:37693 (local address 127.1.132.131:47817)
0923 22:12:25.106615 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.106627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.106670 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.106814 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.106815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.106817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.106887 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.106890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.107546 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.107547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.108408 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.108411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.108965 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.109038 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.109041 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.109084 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.109128 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:25.109228 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.106568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:40513 (local address 127.1.132.131:59987)
0923 22:12:25.106787 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:25.106795 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:25.106843 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:25.106961 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:25.106962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:25.106963 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:25.107029 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.107030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.107620 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.107621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.108433 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:25.108436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.108988 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.108989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:25.109046 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.109049 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:25.109097 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:25.109136 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":55,"thread_start_us":118,"threads_started":1}
I20260923 22:12:25.109587 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a96961a20e64101a8996f4564f457f4" candidate_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" is_pre_election: true
I20260923 22:12:25.109615 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a96961a20e64101a8996f4564f457f4" candidate_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118ba28c875b43c4993c0cf750e7f60b" is_pre_election: true
I20260923 22:12:25.109740 2403 raft_consensus.cc:2454] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 605278fa3ca6411ea8a05d1f6ec711ea in term 0.
I20260923 22:12:25.109755 2269 raft_consensus.cc:2454] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 605278fa3ca6411ea8a05d1f6ec711ea in term 0.
I20260923 22:12:25.109972 2471 leader_election.cc:302] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118ba28c875b43c4993c0cf750e7f60b, 605278fa3ca6411ea8a05d1f6ec711ea; no voters:
I20260923 22:12:25.110138 2608 raft_consensus.cc:2781] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:25.110196 2608 raft_consensus.cc:482] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:25.110216 2608 raft_consensus.cc:3037] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.110862 2608 raft_consensus.cc:504] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.110975 2608 leader_election.cc:288] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [CANDIDATE]: Term 1 election: Requested vote from peers ba251b4d64564c22bb2ba647e3fa87cd (127.1.132.130:37693), 118ba28c875b43c4993c0cf750e7f60b (127.1.132.129:40513)
I20260923 22:12:25.111160 2269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a96961a20e64101a8996f4564f457f4" candidate_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118ba28c875b43c4993c0cf750e7f60b"
I20260923 22:12:25.111207 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a96961a20e64101a8996f4564f457f4" candidate_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba251b4d64564c22bb2ba647e3fa87cd"
I20260923 22:12:25.111236 2269 raft_consensus.cc:3037] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.111269 2403 raft_consensus.cc:3037] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:25.112406 2403 raft_consensus.cc:2454] T 7a96961a20e64101a8996f4564f457f4 P ba251b4d64564c22bb2ba647e3fa87cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 605278fa3ca6411ea8a05d1f6ec711ea in term 1.
I20260923 22:12:25.112453 2269 raft_consensus.cc:2454] T 7a96961a20e64101a8996f4564f457f4 P 118ba28c875b43c4993c0cf750e7f60b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 605278fa3ca6411ea8a05d1f6ec711ea in term 1.
I20260923 22:12:25.112599 2472 leader_election.cc:302] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 605278fa3ca6411ea8a05d1f6ec711ea, ba251b4d64564c22bb2ba647e3fa87cd; no voters:
I20260923 22:12:25.112720 2608 raft_consensus.cc:2781] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:25.112874 2608 raft_consensus.cc:686] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [term 1 LEADER]: Becoming Leader. State: Replica: 605278fa3ca6411ea8a05d1f6ec711ea, State: Running, Role: LEADER
I20260923 22:12:25.113052 2608 consensus_queue.cc:237] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } }
I20260923 22:12:25.113857 2124 catalog_manager.cc:5495] T 7a96961a20e64101a8996f4564f457f4 P 605278fa3ca6411ea8a05d1f6ec711ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 605278fa3ca6411ea8a05d1f6ec711ea (127.1.132.131). New cstate: current_term: 1 leader_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba251b4d64564c22bb2ba647e3fa87cd" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "605278fa3ca6411ea8a05d1f6ec711ea" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "118ba28c875b43c4993c0cf750e7f60b" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40513 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:25.145098 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.137695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.1.132.129:40513)
0923 22:12:25.137762 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.137764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.137847 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:12:25.137922 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.137923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.137932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.137987 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.138598 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.139117 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.139897 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.140093 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.140155 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.142413 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.142428 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.142428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.142436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.143989 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.144411 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.144414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.144415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.144480 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.144622 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.144623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.144623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.144771 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.144876 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.144943 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.145013 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:25.167666 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.160691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.1.132.129:40513)
0923 22:12:25.160737 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.160739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.160790 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:12:25.160842 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.160843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.160852 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.160925 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.161513 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.162030 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.162753 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.162973 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.163031 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.165393 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.165395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.165395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.165403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.166565 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.166861 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.166862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.166862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.166911 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.167158 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.167159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.167159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.167264 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.167343 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.167534 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.167597 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:25.195029 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:25.187330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.1.132.129:40513)
0923 22:12:25.187401 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:25.187404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:25.187499 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:12:25.187551 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:25.187552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:25.187562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:25.187616 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:25.188364 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.188856 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.189910 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:25.190107 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:25.190155 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:25.192740 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:25.192742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:25.192743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:25.192751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:25.193994 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.194320 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.194321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.194321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.194373 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:25.194532 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:25.194533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:25.194533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:25.194641 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:25.194723 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:25.194862 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:12:25.194937 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 22:12:25.195531 2229 tablet_service.cc:2908] Rejecting scan request for tablet 7a96961a20e64101a8996f4564f457f4: Uninitialized: clean time has not yet been initialized
I20260923 22:12:25.196364 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2186
I20260923 22:12:25.201771 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2320
I20260923 22:12:25.207222 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2454
I20260923 22:12:25.212620 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2092
2026-09-23T22:12:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1937 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2624](info): set up 2 sockets
Sep 23 22:12:25 dist-test-slave-l4q5 krb5kdc[2624](info): commencing operation
krb5kdc: starting...
W20260923 22:12:26.171547 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:26.295827 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35691
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:26.375433 2637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:26.375679 2637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:26.375703 2637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:26.376969 2637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:26.377019 2637 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:26.377033 2637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:26.377046 2637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:26.377058 2637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:26.377068 2637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:26.378655 2637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35691
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:26.378885 2637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:26.379070 2637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:26.384334 2643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.384338 2642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.384338 2645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.384361 2637 server_base.cc:1034] running on GCE node
I20260923 22:12:26.384788 2637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:26.385107 2637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:26.386281 2637 hybrid_clock.cc:648] HybridClock initialized: now 1790201546386268 us; error 33 us; skew 500 ppm
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:26.394707 2637 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:26.395208 2637 webserver.cc:460] Webserver started at http://127.1.132.190:36253/ using document root <none> and password file <none>
I20260923 22:12:26.395411 2637 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:26.395453 2637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:26.395547 2637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:26.396755 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "7c2a32b952d24c9fa398738d525ab37e"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c815084376bb10216768bfe8b785d5dd"
server_key_iv: "8eb8e64feba6eee1481682d9628f6a07"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.397094 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "7c2a32b952d24c9fa398738d525ab37e"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c815084376bb10216768bfe8b785d5dd"
server_key_iv: "8eb8e64feba6eee1481682d9628f6a07"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.398468 2637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:26.399230 2652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.399451 2637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:26.399547 2637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "7c2a32b952d24c9fa398738d525ab37e"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c815084376bb10216768bfe8b785d5dd"
server_key_iv: "8eb8e64feba6eee1481682d9628f6a07"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.399610 2637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:26.420709 2637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:26.422674 2637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:26.422817 2637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:26.427644 2637 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35691
I20260923 22:12:26.427700 2714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35691 every 8 connection(s)
I20260923 22:12:26.428046 2637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:26.428709 2715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:26.430795 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2637
I20260923 22:12:26.430893 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:26.431037 1554 external_mini_cluster.cc:1428] Setting key e23f22695c913a0b4d4295c29daffff7
I20260923 22:12:26.431072 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Bootstrap starting.
I20260923 22:12:26.431720 2715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:26.431984 2715 log.cc:824] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:26.432750 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: No bootstrap required, opened a new log
I20260923 22:12:26.434222 2715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } }
I20260923 22:12:26.434353 2715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:26.434374 2715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2a32b952d24c9fa398738d525ab37e, State: Initialized, Role: FOLLOWER
I20260923 22:12:26.434478 2715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } }
I20260923 22:12:26.434549 2715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:26.434576 2715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:26.434618 2715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:26.435267 2715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } }
I20260923 22:12:26.435370 2715 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2a32b952d24c9fa398738d525ab37e; no voters:
I20260923 22:12:26.435518 2715 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:26.435554 2720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:26.435727 2720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [term 1 LEADER]: Becoming Leader. State: Replica: 7c2a32b952d24c9fa398738d525ab37e, State: Running, Role: LEADER
I20260923 22:12:26.435765 2715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:26.437080 2720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } }
I20260923 22:12:26.437642 2721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2a32b952d24c9fa398738d525ab37e. Latest consensus state: current_term: 1 leader_uuid: "7c2a32b952d24c9fa398738d525ab37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } } }
I20260923 22:12:26.437737 2722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2a32b952d24c9fa398738d525ab37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2a32b952d24c9fa398738d525ab37e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35691 } } }
I20260923 22:12:26.437800 2721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:26.437803 2722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:26.438393 2734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
W20260923 22:12:26.438455 2734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:26.438558 2735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:26.438730 2735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:26.440246 2735 catalog_manager.cc:1347] Generated new cluster ID: bb2abbe4f24b4bb79b64f04ee8ed4836
I20260923 22:12:26.440299 2735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:26.442129 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.431551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.1.132.190:35691)
0923 22:12:26.431701 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.431709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.431733 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:12:26.432233 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.432242 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.432307 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.432505 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.433197 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.434669 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.435670 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.436357 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.436482 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.438920 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.438924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.438924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.438934 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.440718 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.441035 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.441038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.441039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.441127 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.441298 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.441299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.441299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.441534 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.441655 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.441777 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.441911 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260923 22:12:26.449548 2735 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:26.450240 2735 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:26.458477 2735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Generated new TSK 0
I20260923 22:12:26.458807 2735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:26.487481 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35691
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:26.569897 2741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:26.570135 2741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:26.570173 2741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:26.571460 2741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:26.571518 2741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:26.571550 2741 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:26.571575 2741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:26.573274 2741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:35691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:26.573560 2741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:26.573787 2741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:26.578881 2741 server_base.cc:1034] running on GCE node
W20260923 22:12:26.578960 2749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.578881 2746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.578884 2747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.579335 2741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:26.579578 2741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:26.580717 2741 hybrid_clock.cc:648] HybridClock initialized: now 1790201546580705 us; error 29 us; skew 500 ppm
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:26.583086 2741 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:26.583555 2741 webserver.cc:460] Webserver started at http://127.1.132.129:33199/ using document root <none> and password file <none>
I20260923 22:12:26.583758 2741 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:26.583804 2741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:26.583925 2741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:26.585006 2741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "f2baeceeec0d4b8283dcee4abb23edab"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "cc4a17de368f3cb8133011cb09eea297"
server_key_iv: "e7710b712f4f31e58a023f57c7fea15c"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.585310 2741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "f2baeceeec0d4b8283dcee4abb23edab"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "cc4a17de368f3cb8133011cb09eea297"
server_key_iv: "e7710b712f4f31e58a023f57c7fea15c"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.586620 2741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:26.587354 2756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.587531 2741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:26.587612 2741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "f2baeceeec0d4b8283dcee4abb23edab"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "cc4a17de368f3cb8133011cb09eea297"
server_key_iv: "e7710b712f4f31e58a023f57c7fea15c"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.587673 2741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:26.609983 2741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:26.612457 2741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:26.612715 2741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:26.613257 2741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:26.613300 2741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.613330 2741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:26.613345 2741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.619675 2741 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:37549
I20260923 22:12:26.619731 2871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:37549 every 8 connection(s)
I20260923 22:12:26.620052 2741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:26.622560 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2741
I20260923 22:12:26.622673 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:26.622838 1554 external_mini_cluster.cc:1428] Setting key e6603df41ca51692391a3be123c488bd
I20260923 22:12:26.623413 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.613706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45091 (local address 127.1.132.190:35691)
0923 22:12:26.613748 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.613750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.613974 (+ 224us) server_negotiation.cc:407] Connection header received
0923 22:12:26.614444 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.614445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.614460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.614528 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.615350 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.616113 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.617617 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.617928 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.618019 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.620689 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.620690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.620691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.620698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.621967 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.622349 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.622350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.622350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.622397 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.622718 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.622719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.622720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.622851 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.622920 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.623233 (+ 313us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.623326 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:26.623577 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.613707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.129:45091)
0923 22:12:26.613852 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.613887 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.614347 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.614604 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.614608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.614943 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.615243 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.615249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.616221 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.616222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.617517 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.617520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.618059 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.618060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.618187 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.618782 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.618793 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.620592 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.622049 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.622055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.622057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.622285 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.622483 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.622484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.622485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.622627 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.622923 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.622930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.623087 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.623205 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.623341 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260923 22:12:26.624097 2872 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35691
I20260923 22:12:26.624208 2872 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:26.624388 2872 heartbeater.cc:511] Master 127.1.132.190:35691 requested a full tablet report, sending...
I20260923 22:12:26.624922 2668 ts_manager.cc:194] Registered new tserver with Master: f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549)
I20260923 22:12:26.625779 2668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:45091
I20260923 22:12:26.630254 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.625105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.129:51803)
0923 22:12:26.625224 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.625233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.625284 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.625393 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.625394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.625575 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.625624 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.625626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.626313 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.626314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.626844 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.626845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.627168 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.627169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.627241 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.627650 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.627656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.627941 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.629326 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.629326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.629327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.629495 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.629662 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.629663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.629663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.629718 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.629945 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.629946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.629994 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.630052 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.630152 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260923 22:12:26.630249 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.625165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51803 (local address 127.1.132.190:35691)
0923 22:12:26.625203 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.625205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.625287 (+ 82us) server_negotiation.cc:407] Connection header received
0923 22:12:26.625337 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.625337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.625347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.625390 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.625687 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.626233 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.626918 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.627113 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.627166 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.628020 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.628021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.628021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.628028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.629258 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.629556 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.629557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.629557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.629612 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.629782 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.629782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.629783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.629891 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.629946 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.630088 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.630159 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:26.650776 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35691
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:26.731215 2881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:26.731426 2881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:26.731457 2881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:26.732666 2881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:26.732705 2881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:26.732733 2881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:26.732754 2881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:26.734167 2881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:35691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:26.734393 2881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:26.734613 2881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:26.739493 2887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.739600 2881 server_base.cc:1034] running on GCE node
W20260923 22:12:26.739678 2886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.739547 2889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.739928 2881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:26.740166 2881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:26.741371 2881 hybrid_clock.cc:648] HybridClock initialized: now 1790201546741348 us; error 32 us; skew 500 ppm
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:26.744058 2881 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:26.744635 2881 webserver.cc:460] Webserver started at http://127.1.132.130:45767/ using document root <none> and password file <none>
I20260923 22:12:26.744882 2881 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:26.744935 2881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:26.745066 2881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:26.746146 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "48d6e932bad7449886c070ad5ff9db08"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "9072c2171000f826ebaadc9db7b4634c"
server_key_iv: "f724476271412c595969f3572102cd6a"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.746500 2881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "48d6e932bad7449886c070ad5ff9db08"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "9072c2171000f826ebaadc9db7b4634c"
server_key_iv: "f724476271412c595969f3572102cd6a"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.747848 2881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:26.748783 2896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.749022 2881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:26.749115 2881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "48d6e932bad7449886c070ad5ff9db08"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "9072c2171000f826ebaadc9db7b4634c"
server_key_iv: "f724476271412c595969f3572102cd6a"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.749184 2881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:26.768000 2881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:26.770653 2881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:26.770810 2881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:26.771345 2881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:26.771387 2881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.771416 2881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:26.771432 2881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.778330 2881 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:42355
I20260923 22:12:26.778370 3011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:42355 every 8 connection(s)
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:26.778754 2881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:26.781752 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.771949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:60605 (local address 127.1.132.190:35691)
0923 22:12:26.772025 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.772027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.772228 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:12:26.772754 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.772757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.772770 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.772839 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.773764 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.774538 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.776096 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.776377 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.776435 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.779028 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.779030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.779030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.779040 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.780453 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.780860 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.780861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.780861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.780912 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.781118 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.781119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.781119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.781234 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.781282 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.781624 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.781686 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:26.781958 2905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.771848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.130:60605)
0923 22:12:26.772088 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.772125 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.772654 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.772951 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.772956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.773341 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.773664 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.773671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.774661 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.774662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.776011 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.776015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.776453 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.776454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.776610 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.777220 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.777230 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.778895 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.780544 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.780552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.780555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.780796 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.780962 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.780963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.780963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.781074 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.781334 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.781340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.781502 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.781603 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.781740 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260923 22:12:26.782388 3012 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35691
I20260923 22:12:26.782461 3012 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:26.782651 3012 heartbeater.cc:511] Master 127.1.132.190:35691 requested a full tablet report, sending...
I20260923 22:12:26.783021 2669 ts_manager.cc:194] Registered new tserver with Master: 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130:42355)
I20260923 22:12:26.783414 2669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:60605
I20260923 22:12:26.785511 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 2881
I20260923 22:12:26.785594 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:26.785785 1554 external_mini_cluster.cc:1428] Setting key ba58e83d3a2ad20cc180f6b79d9e4966
I20260923 22:12:26.788672 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.783435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:45945 (local address 127.1.132.190:35691)
0923 22:12:26.783472 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.783473 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.783545 (+ 72us) server_negotiation.cc:407] Connection header received
0923 22:12:26.783608 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.783609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.783618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.783656 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.783978 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.784488 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.785131 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.785306 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.785371 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.786272 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.786273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.786273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.786280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.787474 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.787860 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.787861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.787862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.787941 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.788116 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.788117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.788117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.788302 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.788362 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.788503 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.788576 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:26.788695 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.783362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.130:45945)
0923 22:12:26.783480 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.783498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.783557 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.783661 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.783661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.783848 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.783914 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.783916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.784574 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.784575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.785065 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.785066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.785366 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.785367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.785416 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.785880 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.785884 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.786204 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.787548 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.787549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.787550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.787771 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.788007 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.788008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.788008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.788066 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.788358 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.788359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.788412 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.788473 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.788578 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:26.810945 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35691
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:26.890740 3021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:26.890971 3021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:26.891005 3021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:26.892292 3021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:26.892342 3021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:26.892371 3021 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:26.892395 3021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:26.893930 3021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:35691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:26.894191 3021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:26.894404 3021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:26.899853 3021 server_base.cc:1034] running on GCE node
W20260923 22:12:26.899833 3029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.899984 3027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:26.899839 3026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:26.900264 3021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:26.900513 3021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:26.901675 3021 hybrid_clock.cc:648] HybridClock initialized: now 1790201546901654 us; error 32 us; skew 500 ppm
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:26.904466 3021 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:26.904923 3021 webserver.cc:460] Webserver started at http://127.1.132.131:38001/ using document root <none> and password file <none>
I20260923 22:12:26.905151 3021 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:26.905198 3021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:26.905309 3021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:26.906486 3021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "26b80fd1d4a144808a65010f79a94629"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c05ea5612808bff73c824bbe695a93d7"
server_key_iv: "d9135c64c155c532345bc683b2f4c02d"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.906857 3021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "26b80fd1d4a144808a65010f79a94629"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c05ea5612808bff73c824bbe695a93d7"
server_key_iv: "d9135c64c155c532345bc683b2f4c02d"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.908262 3021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:26.909128 3036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.909312 3021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:26.909408 3021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "26b80fd1d4a144808a65010f79a94629"
format_stamp: "Formatted at 2026-09-23 22:12:26 on dist-test-slave-l4q5"
server_key: "c05ea5612808bff73c824bbe695a93d7"
server_key_iv: "d9135c64c155c532345bc683b2f4c02d"
server_key_version: "encryptionkey@0"
I20260923 22:12:26.909478 3021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:26.929400 3021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:26.931886 3021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:26.932029 3021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:26.932592 3021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:26.932636 3021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:26.932667 3021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:26.932683 3021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:26.939647 3021 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:36159
I20260923 22:12:26.939742 3151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:36159 every 8 connection(s)
I20260923 22:12:26.940114 3021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:26.942664 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.933101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:50803 (local address 127.1.132.190:35691)
0923 22:12:26.933143 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.933145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.933343 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:12:26.933869 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.933871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.933884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.933966 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.934823 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.935563 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.936843 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.937073 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.937132 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.939780 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.939782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.939782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.939791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.941213 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.941623 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.941624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.941624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.941693 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.941987 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.941988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.941989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.942142 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.942212 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.942526 (+ 314us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.942599 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:26.942830 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.933075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.131:50803)
0923 22:12:26.933228 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.933264 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.933770 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.934068 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.934073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.934426 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.934726 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.934732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.935664 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.935666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.936762 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.936765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.937152 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.937153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.937278 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.937887 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.937895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.939649 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.941293 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.941300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.941302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.941531 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.941778 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.941779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.941779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.941889 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.942226 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.942232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.942372 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.942483 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.942618 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260923 22:12:26.943790 3152 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35691
I20260923 22:12:26.944058 3152 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:26.944404 3152 heartbeater.cc:511] Master 127.1.132.190:35691 requested a full tablet report, sending...
I20260923 22:12:26.944907 2668 ts_manager.cc:194] Registered new tserver with Master: 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159)
I20260923 22:12:26.945408 2668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:50803
I20260923 22:12:26.945909 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3021
I20260923 22:12:26.945983 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:26.946130 1554 external_mini_cluster.cc:1428] Setting key ea748f4b022295dd16a861944370b9fd
I20260923 22:12:26.947166 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:26.950963 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.944655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.1.132.131:40235)
0923 22:12:26.944776 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.944787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.944845 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.944991 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.944992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.945198 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:26.945257 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.945260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.946226 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.946227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.946937 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.946939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.947351 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.947352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.947436 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.947887 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:26.947891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:26.948280 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:26.949927 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.949928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.949928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.950092 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.950314 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:26.950315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:26.950316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:26.950381 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:26.950676 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:26.950677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:26.950736 (+ 59us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.950799 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.950889 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260923 22:12:26.951011 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.944722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:40235 (local address 127.1.132.190:35691)
0923 22:12:26.944769 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.944771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.944830 (+ 59us) server_negotiation.cc:407] Connection header received
0923 22:12:26.944898 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.944899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.944909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.944979 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.945367 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.946084 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.947021 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.947285 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.947349 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.948355 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.948356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.948357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.948364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.949838 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.950168 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.950169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.950170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.950231 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.950471 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.950472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.950472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.950598 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.950659 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.950838 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.950914 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:26 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:26.981415 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.971901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.1.132.190:35691)
0923 22:12:26.971976 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.971978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.971992 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:12:26.972025 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.972027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.972040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.972118 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:26.973114 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.974017 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.975311 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.975669 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.975755 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.978527 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:26.978529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:26.978530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:26.978541 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:26.979992 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.980518 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.980520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.980520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.980592 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:26.980923 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:26.980924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:26.980925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:26.981055 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:26.981137 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.981272 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.981341 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:26.983227 2668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:26.984989 2668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:26.991756 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.1.132.130:42355)
0923 22:12:26.987770 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.987776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.987788 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:26.987848 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.987850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.987863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.987970 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:26.988176 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989318 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.990464 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991307 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991384 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.991460 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.991613 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.991654 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
I20260923 22:12:26.991740 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42355 (local address 127.0.0.1:39072)
0923 22:12:26.987736 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.987746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.987785 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.987968 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.987968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.987970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:26.988032 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.988035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989415 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.989417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.990375 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.990378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991397 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.991495 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.991513 (+ 18us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.991590 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.991649 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":4,"thread_start_us":48,"threads_started":1}
I20260923 22:12:26.991889 3170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37549 (local address 127.0.0.1:35816)
0923 22:12:26.987514 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.987557 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.987623 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.987946 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.987949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.987951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:26.988032 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.988034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989413 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.989414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.990934 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.990937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991709 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.991763 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.991766 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.991795 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.991829 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":45,"thread_start_us":127,"threads_started":1}
I20260923 22:12:26.992035 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.1.132.131:36159)
0923 22:12:26.988065 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.988068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.988078 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:26.988111 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.988113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.988127 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.988255 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:26.988426 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989115 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.990671 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991438 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991496 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.991543 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.991777 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.991819 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":88,"threads_started":1}
I20260923 22:12:26.992211 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:36159 (local address 127.0.0.1:38228)
0923 22:12:26.987725 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:26.987733 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:26.987771 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:26.988311 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:26.988312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:26.988314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:26.988362 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.988364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989199 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.989200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.990586 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:26.990588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991572 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:26.991635 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.991639 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:26.991684 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:26.991729 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260923 22:12:26.992297 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:26.987624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35816 (local address 127.1.132.129:37549)
0923 22:12:26.987773 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:26.987776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:26.987788 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:26.987829 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:26.987832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:26.987848 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:26.987941 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:26.988118 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.989319 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991008 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:26.991610 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:26.991988 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:26.992065 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:26.992092 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:12:26.992139 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260923 22:12:26.992924 3086 tablet_service.cc:1467] Processing CreateTablet for tablet f88416c232164f8e8f72303241fbbebb (DEFAULT_TABLE table=test-table [id=b4828709356b49a6b73ebc34290780c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:26.992947 2806 tablet_service.cc:1467] Processing CreateTablet for tablet f88416c232164f8e8f72303241fbbebb (DEFAULT_TABLE table=test-table [id=b4828709356b49a6b73ebc34290780c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:26.992944 2946 tablet_service.cc:1467] Processing CreateTablet for tablet f88416c232164f8e8f72303241fbbebb (DEFAULT_TABLE table=test-table [id=b4828709356b49a6b73ebc34290780c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:26.993263 3086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88416c232164f8e8f72303241fbbebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:26.993265 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88416c232164f8e8f72303241fbbebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:26.993265 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88416c232164f8e8f72303241fbbebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:26.995944 3175 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Bootstrap starting.
I20260923 22:12:26.996877 3175 tablet_bootstrap.cc:654] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:26.996879 3176 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Bootstrap starting.
I20260923 22:12:26.997098 3177 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Bootstrap starting.
I20260923 22:12:26.997313 3175 log.cc:824] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:26.997555 3176 tablet_bootstrap.cc:654] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:26.997677 3177 tablet_bootstrap.cc:654] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:26.997803 3176 log.cc:824] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:26.997997 3177 log.cc:824] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:26.998236 3175 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: No bootstrap required, opened a new log
I20260923 22:12:26.998298 3175 ts_tablet_manager.cc:1397] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:26.998587 3177 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: No bootstrap required, opened a new log
I20260923 22:12:26.998643 3177 ts_tablet_manager.cc:1397] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:26.999979 3175 raft_consensus.cc:348] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.000011 3177 raft_consensus.cc:348] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.000128 3177 raft_consensus.cc:374] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.000121 3175 raft_consensus.cc:374] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.000147 3177 raft_consensus.cc:729] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2baeceeec0d4b8283dcee4abb23edab, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.000151 3175 raft_consensus.cc:729] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b80fd1d4a144808a65010f79a94629, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.000288 3177 consensus_queue.cc:260] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.000308 3175 consensus_queue.cc:260] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.000550 3177 ts_tablet_manager.cc:1428] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:27.000551 3175 ts_tablet_manager.cc:1428] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:27.000586 2872 heartbeater.cc:503] Master 127.1.132.190:35691 was elected leader, sending a full tablet report...
I20260923 22:12:27.000640 3152 heartbeater.cc:503] Master 127.1.132.190:35691 was elected leader, sending a full tablet report...
I20260923 22:12:27.001199 3176 tablet_bootstrap.cc:492] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: No bootstrap required, opened a new log
I20260923 22:12:27.001276 3176 ts_tablet_manager.cc:1397] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:12:27.002828 3176 raft_consensus.cc:348] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.003003 3176 raft_consensus.cc:374] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.003048 3176 raft_consensus.cc:729] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.003161 3176 consensus_queue.cc:260] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.003449 3176 ts_tablet_manager.cc:1428] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:27.003490 3012 heartbeater.cc:503] Master 127.1.132.190:35691 was elected leader, sending a full tablet report...
W20260923 22:12:27.029457 3013 tablet.cc:2369] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:27.069772 3183 raft_consensus.cc:482] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.069918 3183 raft_consensus.cc:504] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.070269 3183 leader_election.cc:288] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159), f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549)
I20260923 22:12:27.071801 3181 raft_consensus.cc:482] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.071961 3181 raft_consensus.cc:504] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.072918 3181 leader_election.cc:288] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159), 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130:42355)
I20260923 22:12:27.073377 2905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.070476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:36159 (local address 127.1.132.130:39157)
0923 22:12:27.070544 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.070555 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.070604 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.070747 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.070748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.070750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.070813 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.070815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.071420 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.071421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.072400 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.072403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.073104 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.073105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.073199 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.073204 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.073249 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.073291 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:12:27.073598 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.070608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:43629 (local address 127.1.132.129:37549)
0923 22:12:27.070659 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.070661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.070750 (+ 89us) server_negotiation.cc:407] Connection header received
0923 22:12:27.070797 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.070798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.070807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.070852 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.071006 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.071473 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.072686 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.073210 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.073248 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.073256 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.073494 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.073535 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:27.073644 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.070564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:39157 (local address 127.1.132.131:36159)
0923 22:12:27.070605 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.070607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.070617 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:27.070674 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.070674 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.070683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.070728 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.070876 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.071333 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.072494 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.073007 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.073068 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.073077 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.073537 (+ 460us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.073579 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":22}
I20260923 22:12:27.073825 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.070487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37549 (local address 127.1.132.130:43629)
0923 22:12:27.070674 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.070683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.070746 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.070864 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.070864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.070866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.070944 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.070945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.071558 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.071559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.072599 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.072602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.073324 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.073325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.073416 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.073420 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.073697 (+ 277us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.073739 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":114,"threads_started":1}
I20260923 22:12:27.073887 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629" is_pre_election: true
I20260923 22:12:27.074023 3106 raft_consensus.cc:2454] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.074105 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab" is_pre_election: true
I20260923 22:12:27.074199 2826 raft_consensus.cc:2454] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.074256 2900 leader_election.cc:302] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26b80fd1d4a144808a65010f79a94629, 48d6e932bad7449886c070ad5ff9db08; no voters:
I20260923 22:12:27.074419 3183 raft_consensus.cc:2781] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:27.074456 3183 raft_consensus.cc:482] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:27.074473 3183 raft_consensus.cc:3037] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.075157 3183 raft_consensus.cc:504] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.075423 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629"
I20260923 22:12:27.075511 3106 raft_consensus.cc:3037] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.075539 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab"
I20260923 22:12:27.075620 2826 raft_consensus.cc:3037] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.076292 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.072466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42355 (local address 127.1.132.129:33409)
0923 22:12:27.072549 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.072558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.072591 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.072785 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.072786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.072787 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.072878 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.072879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.074375 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.074376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.075401 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.075403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.076042 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.076042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.076117 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.076121 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.076165 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.076226 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":26}
I20260923 22:12:27.076436 3106 raft_consensus.cc:2454] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.076481 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.072657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:33409 (local address 127.1.132.130:42355)
0923 22:12:27.072693 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.072695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.072701 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:27.072717 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.072717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.072725 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.072774 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.072957 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.073922 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.075479 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.075940 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.076003 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.076011 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.076392 (+ 381us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.076430 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:27.076593 2900 leader_election.cc:302] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26b80fd1d4a144808a65010f79a94629, 48d6e932bad7449886c070ad5ff9db08; no voters:
I20260923 22:12:27.076732 2826 raft_consensus.cc:2454] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.076915 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.072467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:36159 (local address 127.1.132.129:54737)
0923 22:12:27.073021 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.073030 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.073077 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.073247 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.073248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.073249 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.073305 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.073306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.074123 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.074124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.075297 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.075299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.076233 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.076234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.076312 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.076316 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.076656 (+ 340us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.076700 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":326,"thread_start_us":91,"threads_started":1}
I20260923 22:12:27.076781 3187 raft_consensus.cc:2781] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:27.077095 3187 raft_consensus.cc:686] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 1 LEADER]: Becoming Leader. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Running, Role: LEADER
I20260923 22:12:27.077124 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.072531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:54737 (local address 127.1.132.131:36159)
0923 22:12:27.072712 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.072713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.073124 (+ 411us) server_negotiation.cc:407] Connection header received
0923 22:12:27.073148 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.073149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.073158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.073211 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.073374 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.074045 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.075361 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.076101 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.076168 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.076175 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.076759 (+ 584us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.076796 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260923 22:12:27.076951 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "f2baeceeec0d4b8283dcee4abb23edab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48d6e932bad7449886c070ad5ff9db08" is_pre_election: true
I20260923 22:12:27.077207 3187 consensus_queue.cc:237] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } }
I20260923 22:12:27.076907 3183 leader_election.cc:288] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Requested vote from peers 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159), f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549)
I20260923 22:12:27.077356 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88416c232164f8e8f72303241fbbebb" candidate_uuid: "f2baeceeec0d4b8283dcee4abb23edab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629" is_pre_election: true
I20260923 22:12:27.077438 3106 raft_consensus.cc:2379] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2baeceeec0d4b8283dcee4abb23edab in current term 1: Already voted for candidate 48d6e932bad7449886c070ad5ff9db08 in this term.
I20260923 22:12:27.077651 2760 leader_election.cc:302] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2baeceeec0d4b8283dcee4abb23edab; no voters: 26b80fd1d4a144808a65010f79a94629, 48d6e932bad7449886c070ad5ff9db08
I20260923 22:12:27.077775 3181 raft_consensus.cc:2726] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 22:12:27.078045 2669 catalog_manager.cc:5495] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:27.085048 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.082526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.1.132.130:42355)
0923 22:12:27.082610 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.082612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.082665 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:12:27.082724 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.082724 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.082739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.082792 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:27.082926 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.083384 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.084153 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.084387 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.084438 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.084667 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:27.084770 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:27.084838 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.084930 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.084948 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:12:27.086860 3106 raft_consensus.cc:1261] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Refusing update from remote peer 48d6e932bad7449886c070ad5ff9db08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:27.086879 2826 raft_consensus.cc:1261] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Refusing update from remote peer 48d6e932bad7449886c070ad5ff9db08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:27.087154 3188 consensus_queue.cc:1035] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.087309 3187 consensus_queue.cc:1035] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.089921 3193 mvcc.cc:205] Tried to move back new op lower bound from 7332665536865075200 to 7332665536828989440. Current Snapshot: MvccSnapshot[applied={T|T < 7332665536865075200}]
I20260923 22:12:27.093423 2669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:27.093533 2669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:27.094261 2669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Sending DeleteTablet for 3 replicas of tablet f88416c232164f8e8f72303241fbbebb
I20260923 22:12:27.094617 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet f88416c232164f8e8f72303241fbbebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:39072
I20260923 22:12:27.094633 3086 tablet_service.cc:1514] Processing DeleteTablet for tablet f88416c232164f8e8f72303241fbbebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:38228
I20260923 22:12:27.094826 3200 tablet_replica.cc:330] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: stopping tablet replica
I20260923 22:12:27.094883 3201 tablet_replica.cc:330] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: stopping tablet replica
I20260923 22:12:27.094919 3200 raft_consensus.cc:2229] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:27.094964 3201 raft_consensus.cc:2229] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:27.094928 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet f88416c232164f8e8f72303241fbbebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:35816
I20260923 22:12:27.095034 3200 raft_consensus.cc:2258] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.095070 3201 raft_consensus.cc:2258] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.095455 3200 ts_tablet_manager.cc:1905] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:27.095505 3201 ts_tablet_manager.cc:1905] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:27.095567 2669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:27.095721 2669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:27.096460 3202 tablet_replica.cc:330] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: stopping tablet replica
I20260923 22:12:27.096658 3202 raft_consensus.cc:2229] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:27.096834 3202 raft_consensus.cc:2258] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.097704 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 (DEFAULT_TABLE table=test-table [id=52083be0d4ea4f5595410f75a1d30894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.097743 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 (DEFAULT_TABLE table=test-table [id=52083be0d4ea4f5595410f75a1d30894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.097829 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f87b0a6458408e95162abb9b6d9c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.097844 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f87b0a6458408e95162abb9b6d9c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.098201 3200 ts_tablet_manager.cc:1918] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:27.098274 3200 log.cc:1196] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/f88416c232164f8e8f72303241fbbebb
I20260923 22:12:27.098519 3200 ts_tablet_manager.cc:1939] T f88416c232164f8e8f72303241fbbebb P 48d6e932bad7449886c070ad5ff9db08: Deleting consensus metadata
I20260923 22:12:27.098927 3176 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Bootstrap starting.
I20260923 22:12:27.099177 3086 tablet_service.cc:1467] Processing CreateTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 (DEFAULT_TABLE table=test-table [id=52083be0d4ea4f5595410f75a1d30894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:27.099280 3086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f87b0a6458408e95162abb9b6d9c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.099290 3176 tablet_bootstrap.cc:654] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.099330 3202 ts_tablet_manager.cc:1905] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:27.099743 2653 catalog_manager.cc:4841] TS 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130:42355): tablet f88416c232164f8e8f72303241fbbebb (table test-table [id=b4828709356b49a6b73ebc34290780c4]) successfully deleted
I20260923 22:12:27.100147 3176 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: No bootstrap required, opened a new log
I20260923 22:12:27.100220 3176 ts_tablet_manager.cc:1397] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:27.100358 3176 raft_consensus.cc:348] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.100430 3176 raft_consensus.cc:374] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.100446 3176 raft_consensus.cc:729] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.100492 3176 consensus_queue.cc:260] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.100589 3176 ts_tablet_manager.cc:1428] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.100797 3202 ts_tablet_manager.cc:1918] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:27.100859 3202 log.cc:1196] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/f88416c232164f8e8f72303241fbbebb
I20260923 22:12:27.101096 3175 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Bootstrap starting.
I20260923 22:12:27.101223 3177 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Bootstrap starting.
I20260923 22:12:27.101457 3175 tablet_bootstrap.cc:654] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.101624 3177 tablet_bootstrap.cc:654] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.101686 3201 ts_tablet_manager.cc:1918] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:27.101753 3201 log.cc:1196] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/f88416c232164f8e8f72303241fbbebb
I20260923 22:12:27.101964 3175 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: No bootstrap required, opened a new log
I20260923 22:12:27.102010 3175 ts_tablet_manager.cc:1397] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:27.102134 3175 raft_consensus.cc:348] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.102190 3175 raft_consensus.cc:374] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.102201 3175 raft_consensus.cc:729] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b80fd1d4a144808a65010f79a94629, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.102237 3175 consensus_queue.cc:260] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.102314 3175 ts_tablet_manager.cc:1428] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.102638 3201 ts_tablet_manager.cc:1939] T f88416c232164f8e8f72303241fbbebb P 26b80fd1d4a144808a65010f79a94629: Deleting consensus metadata
I20260923 22:12:27.103039 3202 ts_tablet_manager.cc:1939] T f88416c232164f8e8f72303241fbbebb P f2baeceeec0d4b8283dcee4abb23edab: Deleting consensus metadata
I20260923 22:12:27.103062 2656 catalog_manager.cc:4841] TS 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159): tablet f88416c232164f8e8f72303241fbbebb (table test-table [id=b4828709356b49a6b73ebc34290780c4]) successfully deleted
I20260923 22:12:27.103413 2654 catalog_manager.cc:4841] TS f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549): tablet f88416c232164f8e8f72303241fbbebb (table test-table [id=b4828709356b49a6b73ebc34290780c4]) successfully deleted
I20260923 22:12:27.103550 3177 tablet_bootstrap.cc:492] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: No bootstrap required, opened a new log
I20260923 22:12:27.103602 3177 ts_tablet_manager.cc:1397] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.103741 3177 raft_consensus.cc:348] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.103814 3177 raft_consensus.cc:374] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.103833 3177 raft_consensus.cc:729] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2baeceeec0d4b8283dcee4abb23edab, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.103874 3177 consensus_queue.cc:260] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.103957 3177 ts_tablet_manager.cc:1428] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260923 22:12:27.120893 2873 tablet.cc:2369] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:12:27.190825 3153 tablet.cc:2369] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:27.236279 3183 raft_consensus.cc:482] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.236384 3183 raft_consensus.cc:504] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.236658 3183 leader_election.cc:288] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549), 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159)
I20260923 22:12:27.236929 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f87b0a6458408e95162abb9b6d9c07" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab" is_pre_election: true
I20260923 22:12:27.236943 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f87b0a6458408e95162abb9b6d9c07" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629" is_pre_election: true
I20260923 22:12:27.237015 2826 raft_consensus.cc:2454] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.237023 3106 raft_consensus.cc:2454] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.237151 2898 leader_election.cc:302] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48d6e932bad7449886c070ad5ff9db08, f2baeceeec0d4b8283dcee4abb23edab; no voters:
I20260923 22:12:27.237263 3183 raft_consensus.cc:2781] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:27.237309 3183 raft_consensus.cc:482] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:27.237327 3183 raft_consensus.cc:3037] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.237951 3183 raft_consensus.cc:504] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.238065 3183 leader_election.cc:288] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Requested vote from peers f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549), 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159)
I20260923 22:12:27.238258 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f87b0a6458408e95162abb9b6d9c07" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629"
I20260923 22:12:27.238265 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f87b0a6458408e95162abb9b6d9c07" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab"
I20260923 22:12:27.238322 3106 raft_consensus.cc:3037] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.238329 2826 raft_consensus.cc:3037] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.238893 3106 raft_consensus.cc:2454] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.239032 2826 raft_consensus.cc:2454] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.239066 2900 leader_election.cc:302] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26b80fd1d4a144808a65010f79a94629, 48d6e932bad7449886c070ad5ff9db08; no voters:
I20260923 22:12:27.239136 3183 raft_consensus.cc:2781] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:27.239180 3183 raft_consensus.cc:686] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 1 LEADER]: Becoming Leader. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Running, Role: LEADER
I20260923 22:12:27.239228 3183 consensus_queue.cc:237] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.239833 2668 catalog_manager.cc:5495] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:12:27 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:27.275835 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.269101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.1.132.190:35691)
0923 22:12:27.269189 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.269191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.269291 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:12:27.269358 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.269359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.269370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.269457 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.269897 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.270393 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.271094 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.271280 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.271333 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.273624 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.273626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.273626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.273634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.274923 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.275251 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.275251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.275252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.275301 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.275444 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.275444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.275445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.275544 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.275589 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.275697 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.275768 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 22:12:27.275961 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.269016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35691 (local address 127.0.0.1:41018)
0923 22:12:27.269215 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.269226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.269299 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.269430 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.269431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.269752 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:27.269817 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.269819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.270486 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.270487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.271017 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.271018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.271337 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.271338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.271388 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.272065 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:27.272075 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:27.273515 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:27.275003 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.275009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.275011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.275194 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.275350 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:27.275350 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:27.275350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:27.275402 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:27.275587 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:27.275592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:27.275632 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.275679 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.275827 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":61,"thread_start_us":85,"threads_started":1}
I20260923 22:12:27.277946 2668 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:41018:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:27.278129 2668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:27.279596 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 147adaa37f174fae8006fe69d1219244 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=065393ebdc044d68a100359c565f9ea8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.279603 3086 tablet_service.cc:1467] Processing CreateTablet for tablet 147adaa37f174fae8006fe69d1219244 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=065393ebdc044d68a100359c565f9ea8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.279623 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 147adaa37f174fae8006fe69d1219244 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=065393ebdc044d68a100359c565f9ea8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:27.279759 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147adaa37f174fae8006fe69d1219244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.279760 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147adaa37f174fae8006fe69d1219244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.279776 3086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147adaa37f174fae8006fe69d1219244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:27.281054 3176 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: Bootstrap starting.
I20260923 22:12:27.281214 3177 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: Bootstrap starting.
I20260923 22:12:27.281222 3175 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: Bootstrap starting.
I20260923 22:12:27.281407 3176 tablet_bootstrap.cc:654] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.282130 3177 tablet_bootstrap.cc:654] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.282141 3175 tablet_bootstrap.cc:654] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:27.282184 3176 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: No bootstrap required, opened a new log
I20260923 22:12:27.282223 3176 ts_tablet_manager.cc:1397] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:27.282336 3176 raft_consensus.cc:348] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.282420 3176 raft_consensus.cc:374] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.282462 3176 raft_consensus.cc:729] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.282505 3176 consensus_queue.cc:260] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.282604 3176 ts_tablet_manager.cc:1428] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:27.282704 3177 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: No bootstrap required, opened a new log
I20260923 22:12:27.282743 3177 ts_tablet_manager.cc:1397] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.282769 3175 tablet_bootstrap.cc:492] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: No bootstrap required, opened a new log
I20260923 22:12:27.282740 3188 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.282810 3175 ts_tablet_manager.cc:1397] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:27.282812 3188 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.282872 3177 raft_consensus.cc:348] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.282931 3177 raft_consensus.cc:374] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.282951 3177 raft_consensus.cc:729] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2baeceeec0d4b8283dcee4abb23edab, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.282912 3175 raft_consensus.cc:348] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.282976 3175 raft_consensus.cc:374] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:27.282994 3175 raft_consensus.cc:729] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b80fd1d4a144808a65010f79a94629, State: Initialized, Role: FOLLOWER
I20260923 22:12:27.282987 3177 consensus_queue.cc:260] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.283033 3175 consensus_queue.cc:260] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.283069 3177 ts_tablet_manager.cc:1428] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.283123 3175 ts_tablet_manager.cc:1428] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:27.283267 3181 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.283341 3181 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.283572 3182 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.283634 3182 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.583290 3182 raft_consensus.cc:482] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.583385 3182 raft_consensus.cc:504] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.583683 3188 raft_consensus.cc:482] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:27.583783 3182 leader_election.cc:288] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549), 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130:42355)
I20260923 22:12:27.583770 3188 raft_consensus.cc:504] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.583937 3188 leader_election.cc:288] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549), 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159)
I20260923 22:12:27.584170 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab" is_pre_election: true
I20260923 22:12:27.584278 2826 raft_consensus.cc:2454] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.584250 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629" is_pre_election: true
I20260923 22:12:27.584332 3106 raft_consensus.cc:2454] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 0.
I20260923 22:12:27.584411 2898 leader_election.cc:302] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48d6e932bad7449886c070ad5ff9db08, f2baeceeec0d4b8283dcee4abb23edab; no voters:
I20260923 22:12:27.584496 3188 raft_consensus.cc:2781] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:27.584542 3188 raft_consensus.cc:482] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:27.584561 3188 raft_consensus.cc:3037] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.585520 3188 raft_consensus.cc:504] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.585657 3188 leader_election.cc:288] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Requested vote from peers f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549), 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159)
I20260923 22:12:27.585843 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab"
I20260923 22:12:27.585935 2826 raft_consensus.cc:3037] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.586634 2826 raft_consensus.cc:2454] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.586789 2898 leader_election.cc:302] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48d6e932bad7449886c070ad5ff9db08, f2baeceeec0d4b8283dcee4abb23edab; no voters:
I20260923 22:12:27.586985 3106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "48d6e932bad7449886c070ad5ff9db08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b80fd1d4a144808a65010f79a94629"
I20260923 22:12:27.587061 3106 raft_consensus.cc:3037] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:27.587587 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.584084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:50895 (local address 127.1.132.130:42355)
0923 22:12:27.584121 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.584122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.584144 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:12:27.584416 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.584417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.584428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.584476 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.584707 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.585399 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.586746 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.587238 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.587285 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.587294 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.587487 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.587524 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260923 22:12:27.587584 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.584124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:33779 (local address 127.1.132.129:37549)
0923 22:12:27.584233 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.584235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.584304 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:12:27.584350 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.584351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.584363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.584414 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:27.584631 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.585519 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.586728 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.587260 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.587329 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.587337 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.587479 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.587519 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":42,"threads_started":1}
I20260923 22:12:27.587584 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.583988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37549 (local address 127.1.132.131:33779)
0923 22:12:27.584244 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.584252 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.584297 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.584465 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.584466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.584469 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.584522 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.584524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.585589 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.585591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.586601 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.586605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.587342 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.587343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.587412 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.587416 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.587463 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.587506 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":48,"threads_started":1}
I20260923 22:12:27.587868 3188 raft_consensus.cc:2781] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:27.587929 3188 raft_consensus.cc:686] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [term 1 LEADER]: Becoming Leader. State: Replica: 48d6e932bad7449886c070ad5ff9db08, State: Running, Role: LEADER
I20260923 22:12:27.587801 3106 raft_consensus.cc:2454] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48d6e932bad7449886c070ad5ff9db08 in term 1.
I20260923 22:12:27.587810 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.583973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42355 (local address 127.1.132.131:50895)
0923 22:12:27.584090 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.584100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.584143 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.584578 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.584579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.584581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:27.584624 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.584626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.585490 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.585492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.586643 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.586645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.587347 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.587348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.587417 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.587420 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.587699 (+ 279us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.587747 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260923 22:12:27.587975 3188 consensus_queue.cc:237] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } }
I20260923 22:12:27.588101 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "26b80fd1d4a144808a65010f79a94629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2baeceeec0d4b8283dcee4abb23edab" is_pre_election: true
I20260923 22:12:27.588222 2826 raft_consensus.cc:2379] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 26b80fd1d4a144808a65010f79a94629 in current term 1: Already voted for candidate 48d6e932bad7449886c070ad5ff9db08 in this term.
I20260923 22:12:27.588235 3183 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: TxnStatusTablet state changed. Reason: New leader 48d6e932bad7449886c070ad5ff9db08. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.588307 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147adaa37f174fae8006fe69d1219244" candidate_uuid: "26b80fd1d4a144808a65010f79a94629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48d6e932bad7449886c070ad5ff9db08" is_pre_election: true
I20260923 22:12:27.588316 3183 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:27.588622 3037 leader_election.cc:302] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26b80fd1d4a144808a65010f79a94629; no voters: 48d6e932bad7449886c070ad5ff9db08, f2baeceeec0d4b8283dcee4abb23edab
I20260923 22:12:27.588716 2669 catalog_manager.cc:5495] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:27.588804 3182 raft_consensus.cc:2726] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 22:12:27.588899 3220 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:27.589015 3220 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:27.639410 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.636926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42355 (local address 127.0.0.1:39094)
0923 22:12:27.637018 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.637030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.637083 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.637218 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.637218 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.637219 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:27.637269 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.637271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.638031 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.638032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.638630 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.638633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.639055 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.639056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.639102 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.639120 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:27.639263 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:27.639267 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.639297 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.639335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260923 22:12:27.639410 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.636989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.1.132.130:42355)
0923 22:12:27.637039 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.637041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.637067 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:27.637135 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.637136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.637148 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.637210 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:27.637306 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.637920 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.638719 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.638989 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.639054 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.639185 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:27.639218 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:27.639260 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.639300 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.639318 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:27.640465 3106 raft_consensus.cc:1261] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Refusing update from remote peer 48d6e932bad7449886c070ad5ff9db08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:27.640482 2826 raft_consensus.cc:1261] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Refusing update from remote peer 48d6e932bad7449886c070ad5ff9db08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:27.640689 3188 consensus_queue.cc:1035] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.640897 3188 consensus_queue.cc:1035] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.641238 3182 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: TxnStatusTablet state changed. Reason: New leader 48d6e932bad7449886c070ad5ff9db08. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.641330 3182 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.641482 3181 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: TxnStatusTablet state changed. Reason: New leader 48d6e932bad7449886c070ad5ff9db08. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.641579 3181 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.642081 3183 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.642167 3223 mvcc.cc:205] Tried to move back new op lower bound from 7332665539133640704 to 7332665538920706048. Current Snapshot: MvccSnapshot[applied={T|T < 7332665539133640704}]
I20260923 22:12:27.642321 3182 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.642417 3182 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 26b80fd1d4a144808a65010f79a94629: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.642542 3224 mvcc.cc:205] Tried to move back new op lower bound from 7332665539133640704 to 7332665538920706048. Current Snapshot: MvccSnapshot[applied={T|T < 7332665539133640704}]
I20260923 22:12:27.642182 3183 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:27.643567 3183 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.643646 3183 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P 48d6e932bad7449886c070ad5ff9db08: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:27.644052 3181 tablet_replica.cc:439] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48d6e932bad7449886c070ad5ff9db08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 } } peers { permanent_uuid: "48d6e932bad7449886c070ad5ff9db08" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42355 } } peers { permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 } } }
I20260923 22:12:27.644138 3181 tablet_replica.cc:442] T 147adaa37f174fae8006fe69d1219244 P f2baeceeec0d4b8283dcee4abb23edab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:27.644627 3188 consensus_queue.cc:1035] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2baeceeec0d4b8283dcee4abb23edab" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.646250 3183 consensus_queue.cc:1035] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26b80fd1d4a144808a65010f79a94629" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:27.648859 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.646284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:51851 (local address 127.1.132.130:42355)
0923 22:12:27.646321 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.646323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.646535 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:12:27.646621 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.646622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.646631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.646656 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:27.646812 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.647356 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.648061 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.648381 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.648455 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.648615 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:27.648644 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:27.648682 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.648756 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.648789 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":18}
I20260923 22:12:27.648937 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.646245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42355 (local address 127.1.132.130:51851)
0923 22:12:27.646465 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:27.646487 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:27.646560 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:27.646709 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:27.646709 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:27.646710 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:27.646750 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.646751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.647422 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.647422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.647973 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:27.647975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.648484 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.648485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:27.648534 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.648551 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:27.648707 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:27.648709 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:27.648754 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:27.648814 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260923 22:12:27.661715 2669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:27.661849 2669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:27.662745 2669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2a32b952d24c9fa398738d525ab37e: Sending DeleteTablet for 3 replicas of tablet 41f87b0a6458408e95162abb9b6d9c07
I20260923 22:12:27.663015 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:39072
I20260923 22:12:27.663213 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:35816
I20260923 22:12:27.663411 3086 tablet_service.cc:1514] Processing DeleteTablet for tablet 41f87b0a6458408e95162abb9b6d9c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:27 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:38228
I20260923 22:12:27.663681 3244 tablet_replica.cc:330] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: stopping tablet replica
I20260923 22:12:27.663708 3242 tablet_replica.cc:330] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: stopping tablet replica
I20260923 22:12:27.663785 3242 raft_consensus.cc:2229] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:27.663844 3242 raft_consensus.cc:2258] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.663863 3244 raft_consensus.cc:2229] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:27.664103 3242 ts_tablet_manager.cc:1905] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:27.664135 3244 raft_consensus.cc:2258] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.664499 3244 ts_tablet_manager.cc:1905] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:27.665894 3242 ts_tablet_manager.cc:1918] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:27.665959 3242 log.cc:1196] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/41f87b0a6458408e95162abb9b6d9c07
I20260923 22:12:27.666188 3242 ts_tablet_manager.cc:1939] T 41f87b0a6458408e95162abb9b6d9c07 P f2baeceeec0d4b8283dcee4abb23edab: Deleting consensus metadata
I20260923 22:12:27.666294 3244 ts_tablet_manager.cc:1918] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:27.666340 3241 tablet_replica.cc:330] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: stopping tablet replica
I20260923 22:12:27.666426 3244 log.cc:1196] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/41f87b0a6458408e95162abb9b6d9c07
I20260923 22:12:27.666414 3241 raft_consensus.cc:2229] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:27.666491 3241 raft_consensus.cc:2258] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:27.666707 3244 ts_tablet_manager.cc:1939] T 41f87b0a6458408e95162abb9b6d9c07 P 26b80fd1d4a144808a65010f79a94629: Deleting consensus metadata
I20260923 22:12:27.666899 2654 catalog_manager.cc:4841] TS f2baeceeec0d4b8283dcee4abb23edab (127.1.132.129:37549): tablet 41f87b0a6458408e95162abb9b6d9c07 (table test-table [id=52083be0d4ea4f5595410f75a1d30894]) successfully deleted
I20260923 22:12:27.667045 2656 catalog_manager.cc:4841] TS 26b80fd1d4a144808a65010f79a94629 (127.1.132.131:36159): tablet 41f87b0a6458408e95162abb9b6d9c07 (table test-table [id=52083be0d4ea4f5595410f75a1d30894]) successfully deleted
I20260923 22:12:27.667145 3241 ts_tablet_manager.cc:1905] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 22:12:27 dist-test-slave-l4q5 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:27.669143 3241 ts_tablet_manager.cc:1918] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:27.669214 3241 log.cc:1196] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/41f87b0a6458408e95162abb9b6d9c07
I20260923 22:12:27.669438 3241 ts_tablet_manager.cc:1939] T 41f87b0a6458408e95162abb9b6d9c07 P 48d6e932bad7449886c070ad5ff9db08: Deleting consensus metadata
I20260923 22:12:27.670572 2653 catalog_manager.cc:4841] TS 48d6e932bad7449886c070ad5ff9db08 (127.1.132.130:42355): tablet 41f87b0a6458408e95162abb9b6d9c07 (table test-table [id=52083be0d4ea4f5595410f75a1d30894]) successfully deleted
I20260923 22:12:27.672497 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.663874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.1.132.129:37549)
0923 22:12:27.663915 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.663918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.664120 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:12:27.664247 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.664249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.664261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.664289 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.664971 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.665663 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.666649 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.666916 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.666962 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.669318 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.669331 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.669331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.669340 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.671283 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.671675 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.671679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.671679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.671742 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.671915 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.671916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.671917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.672068 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.672242 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.672304 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.672390 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:27.672784 2866 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35824
I20260923 22:12:27.680576 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:27.673727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.1.132.190:35691)
0923 22:12:27.673795 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:27.673797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:27.673900 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:12:27.673933 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:27.673934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:27.673947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:27.674019 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:27.674508 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.675275 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.675949 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:27.676286 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:27.676362 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:27.677514 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:27.677516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:27.677516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:27.677525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:27.679178 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.679610 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.679612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.679612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.679682 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:27.679930 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:27.679931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:27.679931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:27.680084 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:27.680229 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:27.680380 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:12:27.680477 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260923 22:12:27.680835 2669 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41020
I20260923 22:12:27.681334 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2741
I20260923 22:12:27.687413 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2881
I20260923 22:12:27.692835 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3021
I20260923 22:12:27.696887 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 2637
2026-09-23T22:12:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2483 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:27 dist-test-slave-l4q5 krb5kdc[3250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:27 dist-test-slave-l4q5 krb5kdc[3250](info): set up 2 sockets
Sep 23 22:12:27 dist-test-slave-l4q5 krb5kdc[3250](info): commencing operation
krb5kdc: starting...
W20260923 22:12:28.655865 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:28 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:28.781391 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34109
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:34109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:28.865610 3263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:28.865857 3263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:28.865891 3263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:28.867161 3263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:28.867219 3263 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:28.867237 3263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:28.867254 3263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:28.867269 3263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:28.867287 3263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:28.869046 3263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:34109
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:28.869346 3263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:28.869611 3263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:28.874729 3271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:28.874840 3268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:28.874909 3269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:28.875042 3263 server_base.cc:1034] running on GCE node
I20260923 22:12:28.875272 3263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:28.875566 3263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:28.876760 3263 hybrid_clock.cc:648] HybridClock initialized: now 1790201548876747 us; error 28 us; skew 500 ppm
Sep 23 22:12:28 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201548, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:28.879460 3263 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:28.879933 3263 webserver.cc:460] Webserver started at http://127.1.132.190:33551/ using document root <none> and password file <none>
I20260923 22:12:28.880141 3263 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:28.880206 3263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:28.880321 3263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:28.881371 3263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "6672b16d8d574af8974ce955e5e8c5d2"
format_stamp: "Formatted at 2026-09-23 22:12:28 on dist-test-slave-l4q5"
server_key: "b2b2fc1b41139e7a8a88f38167a66297"
server_key_iv: "3416a305b330a45c61e57ebd99ad9484"
server_key_version: "encryptionkey@0"
I20260923 22:12:28.881667 3263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "6672b16d8d574af8974ce955e5e8c5d2"
format_stamp: "Formatted at 2026-09-23 22:12:28 on dist-test-slave-l4q5"
server_key: "b2b2fc1b41139e7a8a88f38167a66297"
server_key_iv: "3416a305b330a45c61e57ebd99ad9484"
server_key_version: "encryptionkey@0"
I20260923 22:12:28.882954 3263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:28.883690 3278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:28.883857 3263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:28.883945 3263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "6672b16d8d574af8974ce955e5e8c5d2"
format_stamp: "Formatted at 2026-09-23 22:12:28 on dist-test-slave-l4q5"
server_key: "b2b2fc1b41139e7a8a88f38167a66297"
server_key_iv: "3416a305b330a45c61e57ebd99ad9484"
server_key_version: "encryptionkey@0"
I20260923 22:12:28.884001 3263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:28.908421 3263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:28.910777 3263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:28.910902 3263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:28.916973 3263 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:34109
I20260923 22:12:28.917001 3340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:34109 every 8 connection(s)
I20260923 22:12:28.917374 3263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:28.918077 3341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:28.921854 3341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Bootstrap starting.
I20260923 22:12:28.922521 3341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:28.922842 3341 log.cc:824] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:28.923759 3341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: No bootstrap required, opened a new log
I20260923 22:12:28.926679 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3263
I20260923 22:12:28.926760 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:28.926908 1554 external_mini_cluster.cc:1428] Setting key 9898d6316b39b450a0a2d9ab4d8c48bd
I20260923 22:12:28.926527 3341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } }
I20260923 22:12:28.926959 3341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:28.927014 3341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6672b16d8d574af8974ce955e5e8c5d2, State: Initialized, Role: FOLLOWER
I20260923 22:12:28.927196 3341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } }
I20260923 22:12:28.927287 3341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:28.927321 3341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:28.927364 3341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:28.928076 3341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } }
I20260923 22:12:28.928206 3341 leader_election.cc:302] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6672b16d8d574af8974ce955e5e8c5d2; no voters:
I20260923 22:12:28.928403 3341 leader_election.cc:288] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:28.928440 3346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:28.928541 3346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [term 1 LEADER]: Becoming Leader. State: Replica: 6672b16d8d574af8974ce955e5e8c5d2, State: Running, Role: LEADER
I20260923 22:12:28.928682 3346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } }
I20260923 22:12:28.928684 3341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:28.929468 3348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6672b16d8d574af8974ce955e5e8c5d2. Latest consensus state: current_term: 1 leader_uuid: "6672b16d8d574af8974ce955e5e8c5d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } } }
I20260923 22:12:28.929565 3348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:28.929792 3347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6672b16d8d574af8974ce955e5e8c5d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6672b16d8d574af8974ce955e5e8c5d2" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34109 } } }
I20260923 22:12:28.929906 3347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:28.930019 3355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:28.930933 3355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:28.933145 3355 catalog_manager.cc:1347] Generated new cluster ID: 1905a696c9d14e5eb27d12d607f8466a
I20260923 22:12:28.933211 3355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:28 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:28.938048 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:28.927384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.1.132.190:34109)
0923 22:12:28.927635 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:28.927646 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:28.927680 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:12:28.927907 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:28.927913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:28.927968 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:28.928103 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:28.928854 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.930203 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.931147 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:28.931817 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:28.931946 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:28.934344 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:28.934349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:28.934349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:28.934362 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:28.935937 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:28.936344 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:28.936348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:28.936349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:28.936828 (+ 479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:28.937043 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:28.937044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:28.937045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:28.937324 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:28.937484 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:28.937614 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:12:28.937864 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":120,"threads_started":1}
I20260923 22:12:28.949028 3355 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:28.949796 3355 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:28.962144 3355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Generated new TSK 0
I20260923 22:12:28.962379 3355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:29.001886 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34109
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:29.130215 3367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:29.130442 3367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:29.130472 3367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:29.131729 3367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:29.131772 3367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:29.131793 3367 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:29.131805 3367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:29.131821 3367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:29.133333 3367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:29.133667 3367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:29.133901 3367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:29.138808 3373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:29.138958 3367 server_base.cc:1034] running on GCE node
W20260923 22:12:29.138842 3375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:29.138816 3372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:29.139253 3367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:29.139487 3367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:29.140621 3367 hybrid_clock.cc:648] HybridClock initialized: now 1790201549140604 us; error 32 us; skew 500 ppm
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:29.143069 3367 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:29.143489 3367 webserver.cc:460] Webserver started at http://127.1.132.129:37647/ using document root <none> and password file <none>
I20260923 22:12:29.143693 3367 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:29.143738 3367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:29.143851 3367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:29.144920 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "ba6631f0dbce4088a7899f4517f70a26"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "386b1e933fc0038ef3a20aaa1ed03d36"
server_key_iv: "ddf27571d0079ee48dd81ad3094eb90b"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.145252 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "ba6631f0dbce4088a7899f4517f70a26"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "386b1e933fc0038ef3a20aaa1ed03d36"
server_key_iv: "ddf27571d0079ee48dd81ad3094eb90b"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.146517 3367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:29.147271 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.147428 3367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.147583 3367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "ba6631f0dbce4088a7899f4517f70a26"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "386b1e933fc0038ef3a20aaa1ed03d36"
server_key_iv: "ddf27571d0079ee48dd81ad3094eb90b"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.147668 3367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:29.157480 3367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:29.159626 3367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:29.159744 3367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:29.160293 3367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:29.160337 3367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.160359 3367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:29.160372 3367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.166399 3367 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:34775
I20260923 22:12:29.166457 3497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:34775 every 8 connection(s)
I20260923 22:12:29.166882 3367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:29.168380 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3367
I20260923 22:12:29.168473 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:29.168696 1554 external_mini_cluster.cc:1428] Setting key 124134b915ea29a4d988208034fa171c
I20260923 22:12:29.170631 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.160882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51047 (local address 127.1.132.190:34109)
0923 22:12:29.160944 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.160946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.161140 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:12:29.161745 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.161747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.161765 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.161832 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.162657 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.163418 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.164903 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.165233 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.165317 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.167889 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.167890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.167891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.167898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.169379 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.169744 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.169745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.169746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.169802 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.170000 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.170001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.170002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.170124 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.170179 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.170434 (+ 255us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.170530 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:12:29.170763 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.160852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.129:51047)
0923 22:12:29.161025 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.161062 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.161586 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.161912 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.161917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.162250 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.162545 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.162551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.163533 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.163533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.164818 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.164821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.165324 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.165325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.165511 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.166062 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.166071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.167797 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.169456 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.169462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.169464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.169678 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.169853 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.169854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.169854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.169948 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.170178 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.170184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.170321 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.170406 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.170557 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260923 22:12:29.171388 3498 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34109
I20260923 22:12:29.171507 3498 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:29.171762 3498 heartbeater.cc:511] Master 127.1.132.190:34109 requested a full tablet report, sending...
I20260923 22:12:29.172349 3294 ts_manager.cc:194] Registered new tserver with Master: ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775)
I20260923 22:12:29.173174 3294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:51047
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:12:29.177912 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.172458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58917 (local address 127.1.132.190:34109)
0923 22:12:29.172506 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.172507 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.172593 (+ 86us) server_negotiation.cc:407] Connection header received
0923 22:12:29.172637 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.172638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.172648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.172704 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.173013 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.173741 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.174430 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.174707 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.174775 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.175610 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.175612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.175612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.175620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.177028 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.177347 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.177348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.177348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.177390 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.177527 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.177528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.177528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.177630 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.177678 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.177794 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.177853 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:29.177937 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.172385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.129:58917)
0923 22:12:29.172519 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.172529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.172590 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.172707 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.172707 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.172881 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.172942 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.172943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.173841 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.173842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.174355 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.174357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.174784 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.174785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.174843 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.175258 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.175262 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.175532 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.177129 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.177130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.177130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.177286 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.177425 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.177425 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.177425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.177478 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.177678 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.177679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.177728 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.177779 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.177859 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:29.196635 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34109
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:29.281347 3507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:29.281569 3507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:29.281596 3507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:29.282824 3507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:29.282869 3507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:29.282893 3507 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:29.282904 3507 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:29.282922 3507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:29.284457 3507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:29.284701 3507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:29.284937 3507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:29.290552 3512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:29.290537 3513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:29.290628 3507 server_base.cc:1034] running on GCE node
W20260923 22:12:29.290534 3515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:29.290941 3507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:29.291216 3507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:29.292380 3507 hybrid_clock.cc:648] HybridClock initialized: now 1790201549292364 us; error 32 us; skew 500 ppm
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:29.295042 3507 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:29.295547 3507 webserver.cc:460] Webserver started at http://127.1.132.130:45291/ using document root <none> and password file <none>
I20260923 22:12:29.295770 3507 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:29.295819 3507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:29.295953 3507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:29.297077 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "7dc3faf281114570828c27417a6a1e22"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "4b7d3e1d17768c0bd86bda7953bf2d9e"
server_key_iv: "73047259908dbda3f2f5a802a94712fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.297421 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "7dc3faf281114570828c27417a6a1e22"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "4b7d3e1d17768c0bd86bda7953bf2d9e"
server_key_iv: "73047259908dbda3f2f5a802a94712fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.298720 3507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:29.299531 3522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.299682 3507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.299767 3507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "7dc3faf281114570828c27417a6a1e22"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "4b7d3e1d17768c0bd86bda7953bf2d9e"
server_key_iv: "73047259908dbda3f2f5a802a94712fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.299834 3507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:29.321359 3507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:29.323398 3507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:29.323547 3507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:29.324144 3507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:29.324214 3507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.324245 3507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:29.324262 3507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:29.331013 3507 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:33617
I20260923 22:12:29.331387 3507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:29.331384 3637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:33617 every 8 connection(s)
I20260923 22:12:29.331696 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3507
I20260923 22:12:29.331779 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:29.331940 1554 external_mini_cluster.cc:1428] Setting key 615714373d5ca621f241f053799507b4
I20260923 22:12:29.333475 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.324597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:55929 (local address 127.1.132.190:34109)
0923 22:12:29.324667 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.324670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.324854 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:29.325318 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.325319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.325332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.325396 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.326060 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.326813 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.327912 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.328230 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.328310 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.330587 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.330588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.330589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.330597 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.332053 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.332529 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.332530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.332530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.332597 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.332868 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.332868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.332868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.333000 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.333054 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.333345 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.333411 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:12:29.333715 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.324534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.130:55929)
0923 22:12:29.324731 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.324772 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.325231 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.325479 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.325492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.325760 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.325977 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.325982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.326917 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.326919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.327837 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.327841 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.328331 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.328332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.328478 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.329036 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.329046 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.330492 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.332135 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.332140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.332142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.332451 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.332669 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.332670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.332671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.332809 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.333059 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.333064 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.333224 (+ 160us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.333330 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.333501 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260923 22:12:29.334183 3638 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34109
I20260923 22:12:29.334266 3638 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:29.334441 3638 heartbeater.cc:511] Master 127.1.132.190:34109 requested a full tablet report, sending...
I20260923 22:12:29.334784 3295 ts_manager.cc:194] Registered new tserver with Master: 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:29.335162 3295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:55929
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:12:29.340932 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.335241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:35363 (local address 127.1.132.190:34109)
0923 22:12:29.335324 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.335326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.335388 (+ 62us) server_negotiation.cc:407] Connection header received
0923 22:12:29.335433 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.335434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.335443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.335501 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.335898 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.336459 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.337468 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.337652 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.337704 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.338641 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.338642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.338643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.338648 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.339899 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.340269 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.340270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.340270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.340333 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.340523 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.340524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.340524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.340657 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.340706 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.340812 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.340868 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 22:12:29.340945 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.335187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.130:35363)
0923 22:12:29.335317 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.335328 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.335384 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.335556 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.335557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.335732 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.335806 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.335818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.336551 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.336552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.337345 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.337349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.337721 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.337722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.337796 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.338186 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.338191 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.338559 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.339981 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.339982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.339983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.340172 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.340391 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.340391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.340392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.340467 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.340709 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.340710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.340748 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.340795 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.340872 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:29.359987 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34109
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:29.439031 3647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:29.439241 3647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:29.439275 3647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:29.440500 3647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:29.440542 3647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:29.440573 3647 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:12:29.440595 3647 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:29.440626 3647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:29.442121 3647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:29.442364 3647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:29.442581 3647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:29.447487 3655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:29.447506 3653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:29.447487 3652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:29.447587 3647 server_base.cc:1034] running on GCE node
I20260923 22:12:29.447840 3647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:29.448068 3647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:29.449214 3647 hybrid_clock.cc:648] HybridClock initialized: now 1790201549449198 us; error 32 us; skew 500 ppm
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:29.451834 3647 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:29.452327 3647 webserver.cc:460] Webserver started at http://127.1.132.131:37411/ using document root <none> and password file <none>
I20260923 22:12:29.452548 3647 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:29.452590 3647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:29.452696 3647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:29.453727 3647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "0080f050b1fc4fba935ce512251190a4"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "feea110d4d294a6657f38c945bd07ba0"
server_key_iv: "2c43fbae3fc3b970da4f0b3c0826eefa"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.454130 3647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "0080f050b1fc4fba935ce512251190a4"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "feea110d4d294a6657f38c945bd07ba0"
server_key_iv: "2c43fbae3fc3b970da4f0b3c0826eefa"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.455456 3647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:29.456166 3662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.456353 3647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:29.456429 3647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "0080f050b1fc4fba935ce512251190a4"
format_stamp: "Formatted at 2026-09-23 22:12:29 on dist-test-slave-l4q5"
server_key: "feea110d4d294a6657f38c945bd07ba0"
server_key_iv: "2c43fbae3fc3b970da4f0b3c0826eefa"
server_key_version: "encryptionkey@0"
I20260923 22:12:29.456498 3647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:29.468261 3647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:29.470317 3647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:29.470439 3647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:29.470937 3647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:29.470984 3647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.471007 3647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:29.471019 3647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.477433 3647 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:40915
I20260923 22:12:29.477474 3777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:40915 every 8 connection(s)
I20260923 22:12:29.477793 3647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:29.481050 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.471563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59367 (local address 127.1.132.190:34109)
0923 22:12:29.471620 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.471622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.471752 (+ 130us) server_negotiation.cc:407] Connection header received
0923 22:12:29.472294 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.472296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.472309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.472350 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.473173 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.473985 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.475511 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.475806 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.475894 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.478384 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.478385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.478385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.478393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.479689 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.480135 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.480137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.480137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.480231 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.480485 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.480487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.480487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.480618 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.480680 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.480917 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.480985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:12:29.481241 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.471444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.131:59367)
0923 22:12:29.471622 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.471669 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.472163 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.472441 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.472446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.472781 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.473064 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.473070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.474090 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.474091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.475382 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.475387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.475895 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.475896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.476026 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.476652 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.476664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.478288 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.479808 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.479815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.479817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.480044 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.480300 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.480300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.480301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.480407 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.480678 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.480682 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.480820 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.480900 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.481028 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260923 22:12:29.481729 3778 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34109
I20260923 22:12:29.481815 3778 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:29.482013 3778 heartbeater.cc:511] Master 127.1.132.190:34109 requested a full tablet report, sending...
I20260923 22:12:29.482342 3294 ts_manager.cc:194] Registered new tserver with Master: 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915)
I20260923 22:12:29.482750 3294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:59367
I20260923 22:12:29.484714 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3647
I20260923 22:12:29.484810 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:29.484982 1554 external_mini_cluster.cc:1428] Setting key d4c03b276703604c7dd9a6be71fa518a
I20260923 22:12:29.485790 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:29.488627 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.482753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.1.132.131:51319)
0923 22:12:29.482869 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.482880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.482929 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.483174 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.483175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.483331 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.483388 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.483389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.484041 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.484042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.484626 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.484628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.485093 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.485094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.485167 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.485676 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.485680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.486052 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.487647 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.487648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.487648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.487835 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.488035 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.488036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.488036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.488096 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.488358 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.488358 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.488412 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.488460 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.488544 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260923 22:12:29.488674 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.482826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:51319 (local address 127.1.132.190:34109)
0923 22:12:29.483056 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.483058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.483066 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:29.483086 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.483086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.483106 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.483167 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.483457 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.483948 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.484728 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.485019 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.485094 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.486127 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.486129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.486129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.486136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.487575 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.487915 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.487916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.487917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.487984 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.488145 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.488146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.488146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.488308 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.488370 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.488499 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.488581 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:29.510062 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.502134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.1.132.190:34109)
0923 22:12:29.502224 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.502226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.502236 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:29.502265 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.502266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.502276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.502343 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.503217 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.503797 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.504640 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.504853 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.504920 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.507499 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.507501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.507502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.507509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.508921 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.509275 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.509276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.509277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.509332 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.509576 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.509577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.509577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.509708 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.509796 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.509925 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.509996 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 22:12:29.511924 3294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:29.513569 3294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:29.520134 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.516213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.1.132.130:33617)
0923 22:12:29.516363 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.516367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.516379 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:29.516451 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.516453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.516467 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.516554 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.517029 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.517908 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.518936 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.519701 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.519847 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.519988 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.520031 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260923 22:12:29.520128 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.515838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:33617 (local address 127.0.0.1:34332)
0923 22:12:29.516071 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.516111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.516210 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.516890 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.516891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.516893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.516924 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.516925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.518003 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.518004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.518832 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.518835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.519791 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.519867 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.519882 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.519954 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.520007 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":54,"threads_started":1}
I20260923 22:12:29.520371 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.516011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.1.132.131:40915)
0923 22:12:29.516151 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.516156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.516432 (+ 276us) server_negotiation.cc:407] Connection header received
0923 22:12:29.516564 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.516567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.516585 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.516683 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.516873 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.517610 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519023 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.519730 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519787 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.519847 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.520226 (+ 379us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.520274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260923 22:12:29.520148 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.515839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.0.0.1:58774)
0923 22:12:29.516356 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.516364 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.516400 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.516717 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.516718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.516721 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.516807 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.516810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.517688 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.517689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.518960 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.518961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.519882 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.519961 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.519966 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.520015 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.520058 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":27,"thread_start_us":106,"threads_started":1}
I20260923 22:12:29.520750 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.515841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:34775 (local address 127.0.0.1:36654)
0923 22:12:29.516249 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.516258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.516297 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.516684 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.516688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.516691 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.516808 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.516811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.518942 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.518943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.519653 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.519655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.520529 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.520531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.520586 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.520589 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.520627 (+ 38us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.520659 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260923 22:12:29.520773 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.516047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.1.132.129:34775)
0923 22:12:29.516398 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.516402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.516410 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:29.516448 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.516450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.516467 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.516555 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.517710 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.518843 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.519731 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.520449 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.520510 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.520559 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.520640 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.520678 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":80,"threads_started":1}
I20260923 22:12:29.521358 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 (DEFAULT_TABLE table=test-table [id=965386611ed24288903c726ffad79a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.521371 3432 tablet_service.cc:1467] Processing CreateTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 (DEFAULT_TABLE table=test-table [id=965386611ed24288903c726ffad79a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.521729 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f894dddcf14be9ba69d4ff557771d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.521729 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f894dddcf14be9ba69d4ff557771d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.524695 3572 tablet_service.cc:1467] Processing CreateTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 (DEFAULT_TABLE table=test-table [id=965386611ed24288903c726ffad79a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.525053 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03f894dddcf14be9ba69d4ff557771d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.525074 3801 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Bootstrap starting.
I20260923 22:12:29.525580 3802 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Bootstrap starting.
I20260923 22:12:29.525986 3801 tablet_bootstrap.cc:654] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.526156 3802 tablet_bootstrap.cc:654] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.526336 3801 log.cc:824] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:29.526419 3802 log.cc:824] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:29.527083 3801 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: No bootstrap required, opened a new log
I20260923 22:12:29.527159 3801 ts_tablet_manager.cc:1397] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:29.527238 3802 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: No bootstrap required, opened a new log
I20260923 22:12:29.527341 3802 ts_tablet_manager.cc:1397] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:29.528358 3805 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Bootstrap starting.
I20260923 22:12:29.528903 3805 tablet_bootstrap.cc:654] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.529182 3805 log.cc:824] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:29.529145 3801 raft_consensus.cc:348] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.529343 3801 raft_consensus.cc:374] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.529384 3801 raft_consensus.cc:729] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.529500 3801 consensus_queue.cc:260] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.529608 3802 raft_consensus.cc:348] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.529757 3802 raft_consensus.cc:374] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.529774 3801 ts_tablet_manager.cc:1428] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:29.529798 3802 raft_consensus.cc:729] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.529881 3778 heartbeater.cc:503] Master 127.1.132.190:34109 was elected leader, sending a full tablet report...
I20260923 22:12:29.529951 3805 tablet_bootstrap.cc:492] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: No bootstrap required, opened a new log
I20260923 22:12:29.529973 3802 consensus_queue.cc:260] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.530020 3805 ts_tablet_manager.cc:1397] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:29.530336 3802 ts_tablet_manager.cc:1428] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:12:29.530479 3498 heartbeater.cc:503] Master 127.1.132.190:34109 was elected leader, sending a full tablet report...
I20260923 22:12:29.531842 3805 raft_consensus.cc:348] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.531965 3805 raft_consensus.cc:374] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.531994 3805 raft_consensus.cc:729] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.532104 3805 consensus_queue.cc:260] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.532361 3805 ts_tablet_manager.cc:1428] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:29.532401 3638 heartbeater.cc:503] Master 127.1.132.190:34109 was elected leader, sending a full tablet report...
I20260923 22:12:29.564236 3807 raft_consensus.cc:482] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:29.564379 3807 raft_consensus.cc:504] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.564857 3807 leader_election.cc:288] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:29.567662 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.565233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:60427 (local address 127.1.132.130:33617)
0923 22:12:29.565279 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.565281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.565290 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:29.565317 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.565317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.565326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.565377 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.565597 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.566084 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.566973 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.567346 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.567409 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.567418 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.567558 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.567595 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:29.567662 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.565134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:33617 (local address 127.1.132.131:60427)
0923 22:12:29.565189 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.565200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.565251 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.565441 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.565442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.565445 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.565514 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.565516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.566156 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.566157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.566884 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.566886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.567418 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.567419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.567491 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.567494 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.567539 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.567590 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:29.567790 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.565338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:52369 (local address 127.1.132.129:34775)
0923 22:12:29.565384 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.565387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.565516 (+ 129us) server_negotiation.cc:407] Connection header received
0923 22:12:29.565551 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.565552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.565570 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.565626 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.565807 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.566263 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.567134 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.567487 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.567548 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.567557 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.567692 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.567728 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:29.567795 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.565223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:34775 (local address 127.1.132.131:52369)
0923 22:12:29.565418 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.565428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.565507 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.565666 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.565667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.565669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.565745 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.565746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.566335 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.566336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.567056 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.567058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.567561 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.567562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.567634 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.567637 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.567682 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.567718 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":111,"threads_started":1}
I20260923 22:12:29.568151 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03f894dddcf14be9ba69d4ff557771d9" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22" is_pre_election: true
I20260923 22:12:29.568234 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03f894dddcf14be9ba69d4ff557771d9" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26" is_pre_election: true
I20260923 22:12:29.568346 3452 raft_consensus.cc:2454] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:29.568346 3592 raft_consensus.cc:2454] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:29.568584 3666 leader_election.cc:302] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, 7dc3faf281114570828c27417a6a1e22; no voters:
I20260923 22:12:29.568766 3807 raft_consensus.cc:2781] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:29.568807 3807 raft_consensus.cc:482] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:29.568821 3807 raft_consensus.cc:3037] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.569509 3807 raft_consensus.cc:504] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.569634 3807 leader_election.cc:288] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:29.569828 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03f894dddcf14be9ba69d4ff557771d9" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22"
I20260923 22:12:29.569820 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03f894dddcf14be9ba69d4ff557771d9" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26"
I20260923 22:12:29.569898 3592 raft_consensus.cc:3037] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.569898 3452 raft_consensus.cc:3037] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.570690 3592 raft_consensus.cc:2454] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:29.570691 3452 raft_consensus.cc:2454] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:29.570868 3663 leader_election.cc:302] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:29.570961 3807 raft_consensus.cc:2781] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:29.571106 3807 raft_consensus.cc:686] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 1 LEADER]: Becoming Leader. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Running, Role: LEADER
I20260923 22:12:29.571193 3807 consensus_queue.cc:237] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } }
I20260923 22:12:29.572101 3295 catalog_manager.cc:5495] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0080f050b1fc4fba935ce512251190a4 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:29.581975 3639 tablet.cc:2369] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:12:29 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201548, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:29.607882 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.600984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34109 (local address 127.0.0.1:41190)
0923 22:12:29.601152 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.601163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.601234 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.601350 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.601351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.601690 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:29.601748 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.601750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.602474 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.602492 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.603082 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.603084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.603412 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.603412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.603459 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.603947 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:29.603951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:29.605292 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:29.606758 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.606765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.606767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.606971 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.607146 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:29.607146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:29.607146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:29.607198 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:29.607507 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:29.607514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:29.607565 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.607630 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.607737 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260923 22:12:29.607885 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.601042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.1.132.190:34109)
0923 22:12:29.601103 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.601105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.601244 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:12:29.601274 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.601275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.601286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.601339 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:29.601873 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.602391 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.603173 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.603347 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.603410 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.605399 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:29.605400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:29.605401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:29.605407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:29.606668 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.607036 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.607037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.607037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.607093 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:29.607275 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:29.607276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:29.607277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:29.607425 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:29.607484 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.607657 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.607786 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":36}
I20260923 22:12:29.610250 3295 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:41190:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:29.610476 3295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:29.612960 3572 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6f787b770741dcbda23f558cbd4e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd401c188d64e68ac61df0f1e6a4929]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:29.613087 3432 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6f787b770741dcbda23f558cbd4e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd401c188d64e68ac61df0f1e6a4929]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:29.613234 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6f787b770741dcbda23f558cbd4e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.613240 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6f787b770741dcbda23f558cbd4e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.613518 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6f787b770741dcbda23f558cbd4e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd401c188d64e68ac61df0f1e6a4929]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:29.613675 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6f787b770741dcbda23f558cbd4e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.614599 3802 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: Bootstrap starting.
I20260923 22:12:29.614782 3805 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: Bootstrap starting.
I20260923 22:12:29.615137 3801 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: Bootstrap starting.
I20260923 22:12:29.615283 3802 tablet_bootstrap.cc:654] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.615413 3805 tablet_bootstrap.cc:654] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.615514 3801 tablet_bootstrap.cc:654] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.615991 3802 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: No bootstrap required, opened a new log
I20260923 22:12:29.616037 3802 ts_tablet_manager.cc:1397] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.616070 3805 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: No bootstrap required, opened a new log
I20260923 22:12:29.616077 3801 tablet_bootstrap.cc:492] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: No bootstrap required, opened a new log
I20260923 22:12:29.616109 3805 ts_tablet_manager.cc:1397] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.616112 3801 ts_tablet_manager.cc:1397] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.616209 3802 raft_consensus.cc:348] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616285 3802 raft_consensus.cc:374] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.616305 3802 raft_consensus.cc:729] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.616277 3801 raft_consensus.cc:348] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616277 3805 raft_consensus.cc:348] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616351 3805 raft_consensus.cc:374] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.616351 3801 raft_consensus.cc:374] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.616348 3802 consensus_queue.cc:260] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616372 3805 raft_consensus.cc:729] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.616384 3801 raft_consensus.cc:729] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.616415 3805 consensus_queue.cc:260] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616420 3801 consensus_queue.cc:260] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.616444 3802 ts_tablet_manager.cc:1428] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:29.616496 3801 ts_tablet_manager.cc:1428] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:29.616496 3805 ts_tablet_manager.cc:1428] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.616520 3808 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.616597 3808 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.616631 3809 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.616701 3809 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.616807 3807 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.616870 3807 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.624825 3807 raft_consensus.cc:482] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:29.624914 3807 raft_consensus.cc:504] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.625073 3807 leader_election.cc:288] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:29.625293 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6f787b770741dcbda23f558cbd4e7b" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22" is_pre_election: true
I20260923 22:12:29.625306 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6f787b770741dcbda23f558cbd4e7b" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26" is_pre_election: true
I20260923 22:12:29.625396 3592 raft_consensus.cc:2454] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:29.625396 3452 raft_consensus.cc:2454] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:29.625554 3663 leader_election.cc:302] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:29.625661 3807 raft_consensus.cc:2781] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:29.625705 3807 raft_consensus.cc:482] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:29.625721 3807 raft_consensus.cc:3037] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.626610 3807 raft_consensus.cc:504] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.626742 3807 leader_election.cc:288] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:29.626911 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6f787b770741dcbda23f558cbd4e7b" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22"
I20260923 22:12:29.626911 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6f787b770741dcbda23f558cbd4e7b" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26"
I20260923 22:12:29.626994 3592 raft_consensus.cc:3037] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.626994 3452 raft_consensus.cc:3037] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.627707 3592 raft_consensus.cc:2454] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:29.627701 3452 raft_consensus.cc:2454] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:29.627852 3663 leader_election.cc:302] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:29.627934 3807 raft_consensus.cc:2781] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:29.627977 3807 raft_consensus.cc:686] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [term 1 LEADER]: Becoming Leader. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Running, Role: LEADER
I20260923 22:12:29.628026 3807 consensus_queue.cc:237] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.628347 3811 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: TxnStatusTablet state changed. Reason: New leader 0080f050b1fc4fba935ce512251190a4. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.628448 3811 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:29.628645 3823 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:29.628624 3295 catalog_manager.cc:5495] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0080f050b1fc4fba935ce512251190a4 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:29.628762 3823 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:29.634404 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.631582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.0.0.1:58780)
0923 22:12:29.631671 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.631684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.631752 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.631904 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.631905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.631906 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.631952 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.631954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.632846 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.632847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.633440 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.633442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.633902 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.633903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.633969 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.633986 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.634253 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.634258 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.634291 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.634340 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:12:29.634410 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.631655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.1.132.131:40915)
0923 22:12:29.631700 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.631702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.631729 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:29.631808 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.631809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.631826 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.631902 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.631993 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.632756 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.633518 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.633815 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.633886 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.634079 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.634193 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.634274 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.634296 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.634334 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:29.636528 3592 raft_consensus.cc:1261] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.636559 3452 raft_consensus.cc:1261] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.636893 3811 consensus_queue.cc:1035] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.637102 3807 consensus_queue.cc:1035] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.638216 3809 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: TxnStatusTablet state changed. Reason: New leader 0080f050b1fc4fba935ce512251190a4. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.638226 3808 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: TxnStatusTablet state changed. Reason: New leader 0080f050b1fc4fba935ce512251190a4. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.638329 3809 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.638329 3808 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.639580 3807 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.639662 3807 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:29.639811 3808 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.639896 3808 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P ba6631f0dbce4088a7899f4517f70a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.639871 3811 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.639950 3811 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 0080f050b1fc4fba935ce512251190a4: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:29.640395 3809 tablet_replica.cc:439] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } } }
I20260923 22:12:29.640481 3809 tablet_replica.cc:442] T 9e6f787b770741dcbda23f558cbd4e7b P 7dc3faf281114570828c27417a6a1e22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:29.640549 3828 mvcc.cc:205] Tried to move back new op lower bound from 7332665547308670976 to 7332665547276738560. Current Snapshot: MvccSnapshot[applied={T|T < 7332665547308670976}]
I20260923 22:12:29.641017 3827 mvcc.cc:205] Tried to move back new op lower bound from 7332665547308670976 to 7332665547276738560. Current Snapshot: MvccSnapshot[applied={T|T < 7332665547308670976}]
I20260923 22:12:29.645462 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.642749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.1.132.131:40915)
0923 22:12:29.642788 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.642790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.642799 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:29.642828 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.642829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.642838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.642897 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.642989 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.643740 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.644683 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.645029 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.645105 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.645232 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.645266 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.645311 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.645364 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.645384 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:29.651450 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.648930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:57295 (local address 127.1.132.131:40915)
0923 22:12:29.648980 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.648982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.649260 (+ 278us) server_negotiation.cc:407] Connection header received
0923 22:12:29.649300 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.649301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.649310 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.649366 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.649484 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.649937 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.650802 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.651001 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.651053 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.651235 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.651265 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.651321 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.651342 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.651361 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:12:29.651450 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.649082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.1.132.131:57295)
0923 22:12:29.649176 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.649186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.649250 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.649376 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.649377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.649378 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.649435 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.649436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.650002 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.650003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.650734 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.650737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.651055 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.651056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.651124 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.651139 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.651310 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.651312 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.651343 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.651382 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 22:12:29.653738 3452 raft_consensus.cc:1261] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.653769 3592 raft_consensus.cc:1261] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.653926 3811 consensus_queue.cc:1035] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.654127 3833 consensus_queue.cc:1035] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.655293 3826 mvcc.cc:205] Tried to move back new op lower bound from 7332665547380588544 to 7332665547044421632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665547380588544}]
I20260923 22:12:29.668866 3294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.668991 3294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:29.669781 3294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Sending DeleteTablet for 3 replicas of tablet 03f894dddcf14be9ba69d4ff557771d9
I20260923 22:12:29.670063 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:58774
I20260923 22:12:29.670164 3572 tablet_service.cc:1514] Processing DeleteTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34332
I20260923 22:12:29.670298 3432 tablet_service.cc:1514] Processing DeleteTablet for tablet 03f894dddcf14be9ba69d4ff557771d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:29 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36654
I20260923 22:12:29.670365 3842 tablet_replica.cc:330] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: stopping tablet replica
I20260923 22:12:29.670469 3842 raft_consensus.cc:2229] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.670431 3294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:29.670533 3842 raft_consensus.cc:2258] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:12:29.670560 3294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:29.670825 3843 tablet_replica.cc:330] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: stopping tablet replica
I20260923 22:12:29.670840 3841 tablet_replica.cc:330] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: stopping tablet replica
I20260923 22:12:29.670939 3841 raft_consensus.cc:2229] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:29.671041 3841 raft_consensus.cc:2258] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.671195 3843 raft_consensus.cc:2229] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:29.671304 3843 raft_consensus.cc:2258] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:29.671496 3841 ts_tablet_manager.cc:1905] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.672092 3432 tablet_service.cc:1467] Processing CreateTablet for tablet 5849443cc4f1414081b1844282cba4e7 (DEFAULT_TABLE table=test-table [id=95e862983ae84ddf9f78849038364cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.672235 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5849443cc4f1414081b1844282cba4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.672291 3572 tablet_service.cc:1467] Processing CreateTablet for tablet 5849443cc4f1414081b1844282cba4e7 (DEFAULT_TABLE table=test-table [id=95e862983ae84ddf9f78849038364cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.672390 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5849443cc4f1414081b1844282cba4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.673460 3841 ts_tablet_manager.cc:1918] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.673527 3841 log.cc:1196] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/03f894dddcf14be9ba69d4ff557771d9
I20260923 22:12:29.673642 3802 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Bootstrap starting.
I20260923 22:12:29.673722 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 5849443cc4f1414081b1844282cba4e7 (DEFAULT_TABLE table=test-table [id=95e862983ae84ddf9f78849038364cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:29.673758 3841 ts_tablet_manager.cc:1939] T 03f894dddcf14be9ba69d4ff557771d9 P 0080f050b1fc4fba935ce512251190a4: Deleting consensus metadata
I20260923 22:12:29.673805 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5849443cc4f1414081b1844282cba4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:29.674005 3802 tablet_bootstrap.cc:654] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.674150 3281 catalog_manager.cc:4841] TS 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915): tablet 03f894dddcf14be9ba69d4ff557771d9 (table test-table [id=965386611ed24288903c726ffad79a03]) successfully deleted
I20260923 22:12:29.674593 3802 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: No bootstrap required, opened a new log
I20260923 22:12:29.674644 3802 ts_tablet_manager.cc:1397] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:29.674793 3802 raft_consensus.cc:348] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.674856 3802 raft_consensus.cc:374] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.674875 3802 raft_consensus.cc:729] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.674899 3801 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Bootstrap starting.
I20260923 22:12:29.674916 3802 consensus_queue.cc:260] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.674938 3805 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Bootstrap starting.
I20260923 22:12:29.674998 3802 ts_tablet_manager.cc:1428] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:29.675302 3801 tablet_bootstrap.cc:654] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.675318 3805 tablet_bootstrap.cc:654] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:29.672139 3843 ts_tablet_manager.cc:1905] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.675891 3805 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: No bootstrap required, opened a new log
I20260923 22:12:29.675943 3805 ts_tablet_manager.cc:1397] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:29.676079 3805 raft_consensus.cc:348] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.676151 3805 raft_consensus.cc:374] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.676174 3805 raft_consensus.cc:729] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.676252 3805 consensus_queue.cc:260] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.676281 3842 ts_tablet_manager.cc:1905] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:29.676337 3805 ts_tablet_manager.cc:1428] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:29.677491 3843 ts_tablet_manager.cc:1918] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.677616 3843 log.cc:1196] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/03f894dddcf14be9ba69d4ff557771d9
I20260923 22:12:29.677755 3842 ts_tablet_manager.cc:1918] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:29.677824 3842 log.cc:1196] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/03f894dddcf14be9ba69d4ff557771d9
I20260923 22:12:29.677870 3843 ts_tablet_manager.cc:1939] T 03f894dddcf14be9ba69d4ff557771d9 P ba6631f0dbce4088a7899f4517f70a26: Deleting consensus metadata
I20260923 22:12:29.678049 3842 ts_tablet_manager.cc:1939] T 03f894dddcf14be9ba69d4ff557771d9 P 7dc3faf281114570828c27417a6a1e22: Deleting consensus metadata
I20260923 22:12:29.678403 3282 catalog_manager.cc:4841] TS 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617): tablet 03f894dddcf14be9ba69d4ff557771d9 (table test-table [id=965386611ed24288903c726ffad79a03]) successfully deleted
I20260923 22:12:29.678427 3279 catalog_manager.cc:4841] TS ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775): tablet 03f894dddcf14be9ba69d4ff557771d9 (table test-table [id=965386611ed24288903c726ffad79a03]) successfully deleted
I20260923 22:12:29.680598 3801 tablet_bootstrap.cc:492] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: No bootstrap required, opened a new log
I20260923 22:12:29.680648 3801 ts_tablet_manager.cc:1397] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 22:12:29.680780 3801 raft_consensus.cc:348] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.680845 3801 raft_consensus.cc:374] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:29.680857 3801 raft_consensus.cc:729] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Initialized, Role: FOLLOWER
I20260923 22:12:29.680891 3801 consensus_queue.cc:260] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.680976 3801 ts_tablet_manager.cc:1428] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:29.704339 3808 raft_consensus.cc:482] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:29.704466 3808 raft_consensus.cc:504] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.704811 3808 leader_election.cc:288] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617), 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915)
I20260923 22:12:29.708283 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.705061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:53263 (local address 127.1.132.130:33617)
0923 22:12:29.705106 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.705108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.705259 (+ 151us) server_negotiation.cc:407] Connection header received
0923 22:12:29.705383 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.705383 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.705393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.705421 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.705662 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.706348 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.707537 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.707897 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.707960 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.707969 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.708119 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.708208 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:29.708353 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.705168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45559 (local address 127.1.132.131:40915)
0923 22:12:29.705226 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.705227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.705295 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:12:29.705374 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.705375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.705384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.705435 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:29.705600 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.706107 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.707461 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.707997 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.708078 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.708089 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.708253 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.708297 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:29.708531 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.704995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.1.132.129:45559)
0923 22:12:29.705231 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.705241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.705312 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.705454 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.705455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.705458 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.705529 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.705531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.706185 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.706186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.707352 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.707355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.708081 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.708081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.708142 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.708146 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.708415 (+ 269us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.708461 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260923 22:12:29.708531 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.704995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:33617 (local address 127.1.132.129:53263)
0923 22:12:29.705191 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.705203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.705273 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.705535 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.705536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.705538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:29.705605 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.705607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.706447 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.706448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.707468 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.707470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.707971 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.707972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.708036 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.708040 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.708385 (+ 345us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.708434 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"spinlock_wait_cycles":768,"thread_start_us":66,"threads_started":1}
I20260923 22:12:29.708789 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5849443cc4f1414081b1844282cba4e7" candidate_uuid: "ba6631f0dbce4088a7899f4517f70a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22" is_pre_election: true
I20260923 22:12:29.708881 3592 raft_consensus.cc:2454] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba6631f0dbce4088a7899f4517f70a26 in term 0.
I20260923 22:12:29.708925 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5849443cc4f1414081b1844282cba4e7" candidate_uuid: "ba6631f0dbce4088a7899f4517f70a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080f050b1fc4fba935ce512251190a4" is_pre_election: true
I20260923 22:12:29.709043 3732 raft_consensus.cc:2454] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba6631f0dbce4088a7899f4517f70a26 in term 0.
I20260923 22:12:29.709060 3386 leader_election.cc:302] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dc3faf281114570828c27417a6a1e22, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:29.709214 3808 raft_consensus.cc:2781] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:29.709266 3808 raft_consensus.cc:482] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:29.709285 3808 raft_consensus.cc:3037] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.710203 3808 raft_consensus.cc:504] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.710346 3808 leader_election.cc:288] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [CANDIDATE]: Term 1 election: Requested vote from peers 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617), 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915)
I20260923 22:12:29.710470 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5849443cc4f1414081b1844282cba4e7" candidate_uuid: "ba6631f0dbce4088a7899f4517f70a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22"
I20260923 22:12:29.710552 3592 raft_consensus.cc:3037] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.710528 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5849443cc4f1414081b1844282cba4e7" candidate_uuid: "ba6631f0dbce4088a7899f4517f70a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080f050b1fc4fba935ce512251190a4"
I20260923 22:12:29.710600 3732 raft_consensus.cc:3037] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:29.711349 3592 raft_consensus.cc:2454] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba6631f0dbce4088a7899f4517f70a26 in term 1.
I20260923 22:12:29.711349 3732 raft_consensus.cc:2454] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba6631f0dbce4088a7899f4517f70a26 in term 1.
I20260923 22:12:29.711529 3385 leader_election.cc:302] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, 7dc3faf281114570828c27417a6a1e22, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:29.711648 3808 raft_consensus.cc:2781] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:29.711788 3808 raft_consensus.cc:686] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 1 LEADER]: Becoming Leader. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Running, Role: LEADER
I20260923 22:12:29.711867 3808 consensus_queue.cc:237] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:29.712598 3294 catalog_manager.cc:5495] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129). New cstate: current_term: 1 leader_uuid: "ba6631f0dbce4088a7899f4517f70a26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:29.725594 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.723326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.1.132.129:34775)
0923 22:12:29.723390 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.723392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.723410 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:12:29.723447 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.723447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.723469 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.723525 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.723629 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.724115 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.724837 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.725050 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.725106 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.725243 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.725357 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.725419 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.725470 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.725493 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260923 22:12:29.728302 3779 tablet.cc:2369] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:29.730633 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.728080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:38619 (local address 127.1.132.131:40915)
0923 22:12:29.728127 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.728129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.728252 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:12:29.728314 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.728315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.728325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.728369 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.728591 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.729121 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.729981 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.730193 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.730249 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.730413 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.730447 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.730484 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.730536 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.730555 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:29.730665 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.728032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.1.132.129:38619)
0923 22:12:29.728173 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.728207 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.728267 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.728370 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.728370 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.728372 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.728528 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.728530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.729188 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.729189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.729915 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.729917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.730257 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.730258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.730330 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.730346 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.730496 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.730498 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.730534 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.730578 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260923 22:12:29.732822 3592 raft_consensus.cc:1261] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Refusing update from remote peer ba6631f0dbce4088a7899f4517f70a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.732895 3732 raft_consensus.cc:1261] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Refusing update from remote peer ba6631f0dbce4088a7899f4517f70a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:29.733114 3849 consensus_queue.cc:1035] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.733316 3808 consensus_queue.cc:1035] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:29.741045 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.738400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:34775 (local address 127.1.132.131:51793)
0923 22:12:29.738442 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:29.738453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:29.738487 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:29.738653 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:29.738654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:29.738655 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:29.738700 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.738702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.739342 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.739343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.740260 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:29.740263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.740718 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.740719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:29.740769 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.740774 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.740925 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.740927 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:29.740953 (+ 26us) client_negotiation.cc:240] Negotiation successful
0923 22:12:29.740989 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:12:29.741070 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:29.738433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:51793 (local address 127.1.132.129:34775)
0923 22:12:29.738469 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:29.738471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:29.738479 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:29.738559 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:29.738560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:29.738573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:29.738612 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:29.738751 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.739255 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.740354 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:29.740642 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:29.740703 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:29.740839 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:29.740876 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:29.740917 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:29.740960 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:12:29.740984 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:29.741423 3828 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:12:31.267228 3295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.267333 3295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.268225 3295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Sending DeleteTablet for 3 replicas of tablet 5849443cc4f1414081b1844282cba4e7
I20260923 22:12:31.268497 3432 tablet_service.cc:1514] Processing DeleteTablet for tablet 5849443cc4f1414081b1844282cba4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36654
I20260923 22:12:31.268502 3572 tablet_service.cc:1514] Processing DeleteTablet for tablet 5849443cc4f1414081b1844282cba4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34332
I20260923 22:12:31.268568 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 5849443cc4f1414081b1844282cba4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:58774
I20260923 22:12:31.270670 3880 tablet_replica.cc:330] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: stopping tablet replica
I20260923 22:12:31.270748 3880 raft_consensus.cc:2229] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:31.270787 3880 raft_consensus.cc:2258] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.270891 3881 tablet_replica.cc:330] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: stopping tablet replica
I20260923 22:12:31.271011 3881 raft_consensus.cc:2229] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:31.271042 3880 ts_tablet_manager.cc:1905] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.271121 3881 raft_consensus.cc:2258] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.271364 3881 ts_tablet_manager.cc:1905] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.272737 3880 ts_tablet_manager.cc:1918] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:31.272809 3880 log.cc:1196] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/5849443cc4f1414081b1844282cba4e7
I20260923 22:12:31.273000 3880 ts_tablet_manager.cc:1939] T 5849443cc4f1414081b1844282cba4e7 P 7dc3faf281114570828c27417a6a1e22: Deleting consensus metadata
I20260923 22:12:31.273263 3881 ts_tablet_manager.cc:1918] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:31.273329 3882 tablet_replica.cc:330] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: stopping tablet replica
I20260923 22:12:31.273401 3882 raft_consensus.cc:2229] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:31.273420 3881 log.cc:1196] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/5849443cc4f1414081b1844282cba4e7
I20260923 22:12:31.273473 3882 raft_consensus.cc:2258] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.273499 3282 catalog_manager.cc:4841] TS 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617): tablet 5849443cc4f1414081b1844282cba4e7 (table test-table [id=95e862983ae84ddf9f78849038364cf7]) successfully deleted
I20260923 22:12:31.273695 3881 ts_tablet_manager.cc:1939] T 5849443cc4f1414081b1844282cba4e7 P ba6631f0dbce4088a7899f4517f70a26: Deleting consensus metadata
I20260923 22:12:31.273738 3882 ts_tablet_manager.cc:1905] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.273984 3279 catalog_manager.cc:4841] TS ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775): tablet 5849443cc4f1414081b1844282cba4e7 (table test-table [id=95e862983ae84ddf9f78849038364cf7]) successfully deleted
I20260923 22:12:31.275235 3882 ts_tablet_manager.cc:1918] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:31.275314 3882 log.cc:1196] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/5849443cc4f1414081b1844282cba4e7
I20260923 22:12:31.275503 3882 ts_tablet_manager.cc:1939] T 5849443cc4f1414081b1844282cba4e7 P 0080f050b1fc4fba935ce512251190a4: Deleting consensus metadata
I20260923 22:12:31.275966 3281 catalog_manager.cc:4841] TS 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915): tablet 5849443cc4f1414081b1844282cba4e7 (table test-table [id=95e862983ae84ddf9f78849038364cf7]) successfully deleted
Sep 23 22:12:31 dist-test-slave-l4q5 krb5kdc[3250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:31 dist-test-slave-l4q5 krb5kdc[3250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:31.293304 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.285124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.1.132.190:34109)
0923 22:12:31.285249 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.285251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.285334 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:12:31.285378 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.285379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.285389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.285442 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:31.286069 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.286707 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.287553 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.287760 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.287823 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.290081 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:31.290083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:31.290087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:31.290096 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:31.291670 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:31.292017 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:31.292018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:31.292018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:31.292070 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:31.292264 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:31.292265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:31.292265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:31.292397 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:31.292540 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.293082 (+ 542us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.293191 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260923 22:12:31.294911 3295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:31.295063 3295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:31.296578 3572 tablet_service.cc:1467] Processing CreateTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f (DEFAULT_TABLE table=test-table [id=68d8a6fb670543238d93499596da23f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.296593 3432 tablet_service.cc:1467] Processing CreateTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f (DEFAULT_TABLE table=test-table [id=68d8a6fb670543238d93499596da23f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.296602 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f (DEFAULT_TABLE table=test-table [id=68d8a6fb670543238d93499596da23f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.296720 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b94116f2457e43cbbb7ac7e04a00ed0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.296720 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b94116f2457e43cbbb7ac7e04a00ed0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.296720 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b94116f2457e43cbbb7ac7e04a00ed0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.297755 3891 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Bootstrap starting.
I20260923 22:12:31.298184 3891 tablet_bootstrap.cc:654] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.298301 3893 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Bootstrap starting.
I20260923 22:12:31.298810 3893 tablet_bootstrap.cc:654] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.299005 3891 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: No bootstrap required, opened a new log
I20260923 22:12:31.299058 3891 ts_tablet_manager.cc:1397] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:31.299108 3892 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Bootstrap starting.
I20260923 22:12:31.299208 3891 raft_consensus.cc:348] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.299294 3891 raft_consensus.cc:374] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.299314 3891 raft_consensus.cc:729] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.299337 3893 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: No bootstrap required, opened a new log
I20260923 22:12:31.299364 3891 consensus_queue.cc:260] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.299408 3893 ts_tablet_manager.cc:1397] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:31.299499 3891 ts_tablet_manager.cc:1428] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:31.299525 3893 raft_consensus.cc:348] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.299594 3893 raft_consensus.cc:374] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.299619 3893 raft_consensus.cc:729] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.299683 3893 consensus_queue.cc:260] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.299780 3893 ts_tablet_manager.cc:1428] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:31.300242 3892 tablet_bootstrap.cc:654] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.300849 3892 tablet_bootstrap.cc:492] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: No bootstrap required, opened a new log
I20260923 22:12:31.300904 3892 ts_tablet_manager.cc:1397] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:31.301002 3892 raft_consensus.cc:348] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.301056 3892 raft_consensus.cc:374] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.301069 3892 raft_consensus.cc:729] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.301110 3892 consensus_queue.cc:260] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.301265 3892 ts_tablet_manager.cc:1428] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260923 22:12:31.419080 3499 tablet.cc:2369] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:31.478315 3875 raft_consensus.cc:482] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:31.478416 3875 raft_consensus.cc:504] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.478575 3875 leader_election.cc:288] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:31.478785 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b94116f2457e43cbbb7ac7e04a00ed0f" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26" is_pre_election: true
I20260923 22:12:31.478837 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b94116f2457e43cbbb7ac7e04a00ed0f" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22" is_pre_election: true
I20260923 22:12:31.478917 3452 raft_consensus.cc:2454] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:31.478935 3592 raft_consensus.cc:2454] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 0.
I20260923 22:12:31.479097 3663 leader_election.cc:302] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:31.479210 3875 raft_consensus.cc:2781] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:31.479243 3875 raft_consensus.cc:482] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:31.479255 3875 raft_consensus.cc:3037] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.480031 3875 raft_consensus.cc:504] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.480216 3875 leader_election.cc:288] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Requested vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617)
I20260923 22:12:31.480353 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b94116f2457e43cbbb7ac7e04a00ed0f" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26"
I20260923 22:12:31.480445 3452 raft_consensus.cc:3037] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.480525 3592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b94116f2457e43cbbb7ac7e04a00ed0f" candidate_uuid: "0080f050b1fc4fba935ce512251190a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc3faf281114570828c27417a6a1e22"
I20260923 22:12:31.480778 3592 raft_consensus.cc:3037] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.481134 3452 raft_consensus.cc:2454] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:31.481312 3663 leader_election.cc:302] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0080f050b1fc4fba935ce512251190a4, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:31.481395 3875 raft_consensus.cc:2781] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:31.481460 3875 raft_consensus.cc:686] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 1 LEADER]: Becoming Leader. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Running, Role: LEADER
I20260923 22:12:31.481535 3592 raft_consensus.cc:2454] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0080f050b1fc4fba935ce512251190a4 in term 1.
I20260923 22:12:31.481516 3875 consensus_queue.cc:237] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.482154 3294 catalog_manager.cc:5495] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0080f050b1fc4fba935ce512251190a4 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "0080f050b1fc4fba935ce512251190a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:31.531184 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.529000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.1.132.131:40915)
0923 22:12:31.529167 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.529169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.529177 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:31.529198 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.529199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.529211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.529269 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:31.529384 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.529877 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.530681 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.530868 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.530914 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.531009 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.531034 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.531064 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.531093 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.531107 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260923 22:12:31.533196 3592 raft_consensus.cc:1261] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:31.533252 3452 raft_consensus.cc:1261] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Refusing update from remote peer 0080f050b1fc4fba935ce512251190a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:31.533401 3858 consensus_queue.cc:1035] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:31.533493 3875 consensus_queue.cc:1035] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:31.543466 3293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.543576 3293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:31.544277 3293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Sending DeleteTablet for 3 replicas of tablet b94116f2457e43cbbb7ac7e04a00ed0f
I20260923 22:12:31.544528 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:58774
I20260923 22:12:31.544585 3432 tablet_service.cc:1514] Processing DeleteTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36654
I20260923 22:12:31.544631 3882 tablet_replica.cc:330] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: stopping tablet replica
I20260923 22:12:31.544679 3882 raft_consensus.cc:2229] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:31.544678 3572 tablet_service.cc:1514] Processing DeleteTablet for tablet b94116f2457e43cbbb7ac7e04a00ed0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:31 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34332
I20260923 22:12:31.544752 3882 raft_consensus.cc:2258] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.544775 3881 tablet_replica.cc:330] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: stopping tablet replica
I20260923 22:12:31.544826 3881 raft_consensus.cc:2229] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:31.544866 3881 raft_consensus.cc:2258] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.544795 3293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:31.544931 3880 tablet_replica.cc:330] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: stopping tablet replica
I20260923 22:12:31.544979 3880 raft_consensus.cc:2229] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 22:12:31.544974 3293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:31.545022 3880 raft_consensus.cc:2258] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:31.545370 3880 ts_tablet_manager.cc:1905] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.546485 3881 ts_tablet_manager.cc:1905] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.546744 3880 ts_tablet_manager.cc:1918] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:31.546806 3880 log.cc:1196] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/b94116f2457e43cbbb7ac7e04a00ed0f
I20260923 22:12:31.546983 3880 ts_tablet_manager.cc:1939] T b94116f2457e43cbbb7ac7e04a00ed0f P 7dc3faf281114570828c27417a6a1e22: Deleting consensus metadata
I20260923 22:12:31.547267 3282 catalog_manager.cc:4841] TS 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617): tablet b94116f2457e43cbbb7ac7e04a00ed0f (table test-table [id=68d8a6fb670543238d93499596da23f8]) successfully deleted
I20260923 22:12:31.547727 3881 ts_tablet_manager.cc:1918] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:31.547776 3881 log.cc:1196] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/b94116f2457e43cbbb7ac7e04a00ed0f
I20260923 22:12:31.547943 3881 ts_tablet_manager.cc:1939] T b94116f2457e43cbbb7ac7e04a00ed0f P ba6631f0dbce4088a7899f4517f70a26: Deleting consensus metadata
I20260923 22:12:31.548306 3882 ts_tablet_manager.cc:1905] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:31.548985 3572 tablet_service.cc:1467] Processing CreateTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 (DEFAULT_TABLE table=test-table [id=c2004880593a4768ae8e7ad62e0cbd86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.548998 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 (DEFAULT_TABLE table=test-table [id=c2004880593a4768ae8e7ad62e0cbd86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.549098 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831602ebbe14c5d9ddad551fb49c736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.549177 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831602ebbe14c5d9ddad551fb49c736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.549409 3432 tablet_service.cc:1467] Processing CreateTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 (DEFAULT_TABLE table=test-table [id=c2004880593a4768ae8e7ad62e0cbd86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:31.549587 3432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0831602ebbe14c5d9ddad551fb49c736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:31.550261 3893 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Bootstrap starting.
I20260923 22:12:31.550565 3893 tablet_bootstrap.cc:654] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.550774 3891 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26: Bootstrap starting.
I20260923 22:12:31.550806 3892 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Bootstrap starting.
I20260923 22:12:31.551090 3892 tablet_bootstrap.cc:654] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.551106 3891 tablet_bootstrap.cc:654] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:31.551329 3279 catalog_manager.cc:4841] TS ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775): tablet b94116f2457e43cbbb7ac7e04a00ed0f (table test-table [id=68d8a6fb670543238d93499596da23f8]) successfully deleted
I20260923 22:12:31.551596 3891 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26: No bootstrap required, opened a new log
I20260923 22:12:31.551606 3892 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: No bootstrap required, opened a new log
I20260923 22:12:31.551648 3892 ts_tablet_manager.cc:1397] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:31.551649 3891 ts_tablet_manager.cc:1397] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:31.551790 3892 raft_consensus.cc:348] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.551788 3891 raft_consensus.cc:348] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.551860 3892 raft_consensus.cc:374] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.551862 3891 raft_consensus.cc:374] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.551882 3892 raft_consensus.cc:729] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.551882 3891 raft_consensus.cc:729] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6631f0dbce4088a7899f4517f70a26, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.551923 3892 consensus_queue.cc:260] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.551923 3891 consensus_queue.cc:260] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.552019 3892 ts_tablet_manager.cc:1428] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:31.552019 3891 ts_tablet_manager.cc:1428] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:31.552030 3882 ts_tablet_manager.cc:1918] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:31.552076 3882 log.cc:1196] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/b94116f2457e43cbbb7ac7e04a00ed0f
I20260923 22:12:31.552280 3882 ts_tablet_manager.cc:1939] T b94116f2457e43cbbb7ac7e04a00ed0f P 0080f050b1fc4fba935ce512251190a4: Deleting consensus metadata
I20260923 22:12:31.552767 3893 tablet_bootstrap.cc:492] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: No bootstrap required, opened a new log
I20260923 22:12:31.552819 3893 ts_tablet_manager.cc:1397] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:12:31.552819 3281 catalog_manager.cc:4841] TS 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915): tablet b94116f2457e43cbbb7ac7e04a00ed0f (table test-table [id=68d8a6fb670543238d93499596da23f8]) successfully deleted
I20260923 22:12:31.552934 3893 raft_consensus.cc:348] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.553002 3893 raft_consensus.cc:374] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:31.553020 3893 raft_consensus.cc:729] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0080f050b1fc4fba935ce512251190a4, State: Initialized, Role: FOLLOWER
I20260923 22:12:31.553056 3893 consensus_queue.cc:260] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.553143 3893 ts_tablet_manager.cc:1428] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:31.603533 3897 raft_consensus.cc:482] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:31.603657 3897 raft_consensus.cc:504] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.604007 3897 leader_election.cc:288] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915)
I20260923 22:12:31.607003 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.604407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:33385 (local address 127.1.132.131:40915)
0923 22:12:31.604510 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.604512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.604519 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:31.604545 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.604545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.604553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.604599 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:31.604773 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.605243 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606277 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.606666 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606731 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.606742 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.606883 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.606933 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:31.607005 3909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.604304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:39923 (local address 127.1.132.129:34775)
0923 22:12:31.604513 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.604515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.604525 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:31.604578 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.604579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.604636 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.604726 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:31.604887 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.605356 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606278 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.606663 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606735 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.606744 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.606881 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.606923 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260923 22:12:31.607033 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.604265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:34775 (local address 127.1.132.130:39923)
0923 22:12:31.604441 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.604451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:31.604514 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:31.604750 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:31.604751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:31.604754 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:31.604822 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.604823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.605448 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.605449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.606184 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.606186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.606737 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.606815 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.606819 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:31.606867 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:31.606912 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":105,"threads_started":1}
I20260923 22:12:31.607038 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.604230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.1.132.130:33385)
0923 22:12:31.604367 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.604378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:31.604436 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:31.604619 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:31.604620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:31.604623 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:31.604695 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.604697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.605320 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.605321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.606181 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.606183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.606746 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.606747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.606818 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.606822 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:31.606869 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:31.606915 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260923 22:12:31.607450 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831602ebbe14c5d9ddad551fb49c736" candidate_uuid: "7dc3faf281114570828c27417a6a1e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080f050b1fc4fba935ce512251190a4" is_pre_election: true
I20260923 22:12:31.607492 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831602ebbe14c5d9ddad551fb49c736" candidate_uuid: "7dc3faf281114570828c27417a6a1e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26" is_pre_election: true
I20260923 22:12:31.607601 3732 raft_consensus.cc:2454] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dc3faf281114570828c27417a6a1e22 in term 0.
I20260923 22:12:31.607601 3452 raft_consensus.cc:2454] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dc3faf281114570828c27417a6a1e22 in term 0.
I20260923 22:12:31.607801 3523 leader_election.cc:302] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dc3faf281114570828c27417a6a1e22, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:31.607993 3897 raft_consensus.cc:2781] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:31.608050 3897 raft_consensus.cc:482] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:31.608072 3897 raft_consensus.cc:3037] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.609265 3897 raft_consensus.cc:504] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.609488 3897 leader_election.cc:288] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [CANDIDATE]: Term 1 election: Requested vote from peers ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775), 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915)
I20260923 22:12:31.609766 3452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831602ebbe14c5d9ddad551fb49c736" candidate_uuid: "7dc3faf281114570828c27417a6a1e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6631f0dbce4088a7899f4517f70a26"
I20260923 22:12:31.609762 3732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0831602ebbe14c5d9ddad551fb49c736" candidate_uuid: "7dc3faf281114570828c27417a6a1e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0080f050b1fc4fba935ce512251190a4"
I20260923 22:12:31.609861 3732 raft_consensus.cc:3037] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.609861 3452 raft_consensus.cc:3037] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:31.610543 3452 raft_consensus.cc:2454] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dc3faf281114570828c27417a6a1e22 in term 1.
I20260923 22:12:31.610778 3523 leader_election.cc:302] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dc3faf281114570828c27417a6a1e22, ba6631f0dbce4088a7899f4517f70a26; no voters:
I20260923 22:12:31.610855 3732 raft_consensus.cc:2454] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dc3faf281114570828c27417a6a1e22 in term 1.
I20260923 22:12:31.610890 3897 raft_consensus.cc:2781] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:31.611068 3897 raft_consensus.cc:686] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 1 LEADER]: Becoming Leader. State: Replica: 7dc3faf281114570828c27417a6a1e22, State: Running, Role: LEADER
I20260923 22:12:31.611155 3897 consensus_queue.cc:237] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } }
I20260923 22:12:31.611996 3293 catalog_manager.cc:5495] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dc3faf281114570828c27417a6a1e22 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "7dc3faf281114570828c27417a6a1e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dc3faf281114570828c27417a6a1e22" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 33617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:31.629592 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.627106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.1.132.130:33617)
0923 22:12:31.627261 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.627262 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.627267 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:12:31.627300 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.627300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.627319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.627389 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:31.627497 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.627995 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.628754 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.628982 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.629039 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.629180 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.629339 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.629444 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.629468 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.629491 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260923 22:12:31.635138 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.632642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40915 (local address 127.1.132.130:57431)
0923 22:12:31.632858 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.632869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:31.632937 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:31.633061 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:31.633063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:31.633064 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:31.633140 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.633141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.633727 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.633728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.634366 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.634368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.634723 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.634724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.634786 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.634801 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.634944 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.634945 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:31.634982 (+ 37us) client_negotiation.cc:240] Negotiation successful
0923 22:12:31.635034 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
I20260923 22:12:31.635128 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.632738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:57431 (local address 127.1.132.131:40915)
0923 22:12:31.632819 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.632820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.632928 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:12:31.632996 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.632997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.633015 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.633055 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:31.633194 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.633657 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.634458 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.634645 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.634701 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.634868 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.634898 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.634937 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.635015 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.635038 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:12:31.637282 3732 raft_consensus.cc:1261] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Refusing update from remote peer 7dc3faf281114570828c27417a6a1e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:31.637287 3452 raft_consensus.cc:1261] T 0831602ebbe14c5d9ddad551fb49c736 P ba6631f0dbce4088a7899f4517f70a26 [term 1 FOLLOWER]: Refusing update from remote peer 7dc3faf281114570828c27417a6a1e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:31.637555 3910 consensus_queue.cc:1035] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0080f050b1fc4fba935ce512251190a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:31.637686 3897 consensus_queue.cc:1035] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6631f0dbce4088a7899f4517f70a26" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:31.645943 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.643389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:45279 (local address 127.1.132.130:33617)
0923 22:12:31.643427 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:31.643430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:31.643599 (+ 169us) server_negotiation.cc:407] Connection header received
0923 22:12:31.643625 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:31.643625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:31.643635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:31.643687 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:31.643770 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.644262 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.645077 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.645304 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.645356 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.645508 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.645568 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.645603 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:31.645853 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 22:12:31.645880 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:31.645984 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:31.643347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:33617 (local address 127.1.132.131:45279)
0923 22:12:31.643480 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:31.643491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:31.643545 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:31.643687 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:31.643688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:31.643689 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:31.643736 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.643737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.644316 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.644317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.644994 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:31.644997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:31.645390 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:31.645391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:31.645442 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:31.645447 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:31.645696 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:31.645702 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:12:31.645815 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:12:31.645892 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260923 22:12:31.646539 3864 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:12:33.171459 3295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:33.171563 3295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:33.172478 3295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6672b16d8d574af8974ce955e5e8c5d2: Sending DeleteTablet for 3 replicas of tablet 0831602ebbe14c5d9ddad551fb49c736
I20260923 22:12:33.172724 3432 tablet_service.cc:1514] Processing DeleteTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:33 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36654
I20260923 22:12:33.172777 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:33 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:58774
I20260923 22:12:33.172777 3572 tablet_service.cc:1514] Processing DeleteTablet for tablet 0831602ebbe14c5d9ddad551fb49c736 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:33 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34332
I20260923 22:12:33.172857 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3367
I20260923 22:12:33.173615 3945 tablet_replica.cc:330] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: stopping tablet replica
I20260923 22:12:33.173673 3945 raft_consensus.cc:2229] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:33.173710 3945 raft_consensus.cc:2258] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:33.173974 3945 ts_tablet_manager.cc:1905] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:33.175241 3944 tablet_replica.cc:330] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: stopping tablet replica
I20260923 22:12:33.175305 3944 raft_consensus.cc:2229] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:33.175416 3944 raft_consensus.cc:2258] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:33.175580 3945 ts_tablet_manager.cc:1918] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:33.175652 3945 log.cc:1196] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/0831602ebbe14c5d9ddad551fb49c736
I20260923 22:12:33.175702 3944 ts_tablet_manager.cc:1905] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:33.175863 3945 ts_tablet_manager.cc:1939] T 0831602ebbe14c5d9ddad551fb49c736 P 0080f050b1fc4fba935ce512251190a4: Deleting consensus metadata
I20260923 22:12:33.176224 3281 catalog_manager.cc:4841] TS 0080f050b1fc4fba935ce512251190a4 (127.1.132.131:40915): tablet 0831602ebbe14c5d9ddad551fb49c736 (table test-table [id=c2004880593a4768ae8e7ad62e0cbd86]) successfully deleted
I20260923 22:12:33.177459 3944 ts_tablet_manager.cc:1918] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:12:33.177549 3944 log.cc:1196] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/0831602ebbe14c5d9ddad551fb49c736
I20260923 22:12:33.177939 3944 ts_tablet_manager.cc:1939] T 0831602ebbe14c5d9ddad551fb49c736 P 7dc3faf281114570828c27417a6a1e22: Deleting consensus metadata
I20260923 22:12:33.178396 3282 catalog_manager.cc:4841] TS 7dc3faf281114570828c27417a6a1e22 (127.1.132.130:33617): tablet 0831602ebbe14c5d9ddad551fb49c736 (table test-table [id=c2004880593a4768ae8e7ad62e0cbd86]) successfully deleted
W20260923 22:12:33.179430 3279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:34775 (error 108)
I20260923 22:12:33.179601 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3507
I20260923 22:12:33.182118 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:33.181878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:34775 (local address 127.0.0.1:36670)
0923 22:12:33.182019 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:33.182074 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
W20260923 22:12:33.182256 3279 catalog_manager.cc:4584] TS ba6631f0dbce4088a7899f4517f70a26 (127.1.132.129:34775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0831602ebbe14c5d9ddad551fb49c736: Network error: Client connection negotiation failed: client connection to 127.1.132.129:34775: connect: Connection refused (error 111)
I20260923 22:12:33.185374 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3647
I20260923 22:12:33.191545 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3263
2026-09-23T22:12:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5496 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:33 dist-test-slave-l4q5 krb5kdc[3950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:33 dist-test-slave-l4q5 krb5kdc[3950](info): set up 2 sockets
Sep 23 22:12:33 dist-test-slave-l4q5 krb5kdc[3950](info): commencing operation
krb5kdc: starting...
W20260923 22:12:34.167810 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:34.300536 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:38871
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:38871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:34.386089 3963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:34.386324 3963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:34.386348 3963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:34.387632 3963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:34.387689 3963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:34.387703 3963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:34.387715 3963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:34.387727 3963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:34.389390 3963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:38871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:38871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:34.389622 3963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:34.389807 3963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:34.394814 3969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.394819 3971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.394814 3968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.395066 3963 server_base.cc:1034] running on GCE node
I20260923 22:12:34.395370 3963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:34.395686 3963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:34.396854 3963 hybrid_clock.cc:648] HybridClock initialized: now 1790201554396839 us; error 31 us; skew 500 ppm
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:34.399369 3963 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:34.399904 3963 webserver.cc:460] Webserver started at http://127.1.132.190:32891/ using document root <none> and password file <none>
I20260923 22:12:34.400139 3963 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:34.400216 3963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:34.400348 3963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:34.401446 3963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "b7527384aa90429a843eb2cc8f7d9409"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f96bc47f033731e958be60f84cb63c4"
server_key_iv: "6d00973b47d1f9e1629b6105ae619e54"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.401815 3963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "b7527384aa90429a843eb2cc8f7d9409"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f96bc47f033731e958be60f84cb63c4"
server_key_iv: "6d00973b47d1f9e1629b6105ae619e54"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.403177 3963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:34.403981 3978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.404206 3963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:34.404301 3963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "b7527384aa90429a843eb2cc8f7d9409"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f96bc47f033731e958be60f84cb63c4"
server_key_iv: "6d00973b47d1f9e1629b6105ae619e54"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.404372 3963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:34.426075 3963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:34.428171 3963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:34.428363 3963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:34.432232 3963 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:38871
I20260923 22:12:34.432261 4030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:38871 every 8 connection(s)
I20260923 22:12:34.432607 3963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:34.433461 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:34.435684 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 3963
I20260923 22:12:34.435771 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:34.435824 4031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409: Bootstrap starting.
I20260923 22:12:34.435931 1554 external_mini_cluster.cc:1428] Setting key 65bc966dda195934bfa1cc25aee149ee
I20260923 22:12:34.436566 4031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:34.436870 4031 log.cc:824] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:34.437678 4031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409: No bootstrap required, opened a new log
I20260923 22:12:34.439889 4031 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } }
I20260923 22:12:34.440100 4031 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:34.440137 4031 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7527384aa90429a843eb2cc8f7d9409, State: Initialized, Role: FOLLOWER
I20260923 22:12:34.440307 4031 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } }
I20260923 22:12:34.440388 4031 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:34.440411 4031 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:34.440459 4031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:34.441191 4031 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } }
I20260923 22:12:34.441285 4031 leader_election.cc:302] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7527384aa90429a843eb2cc8f7d9409; no voters:
I20260923 22:12:34.441458 4031 leader_election.cc:288] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:34.441519 4036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:34.441682 4036 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [term 1 LEADER]: Becoming Leader. State: Replica: b7527384aa90429a843eb2cc8f7d9409, State: Running, Role: LEADER
I20260923 22:12:34.441746 4031 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:34.441784 4036 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } }
I20260923 22:12:34.442160 4037 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7527384aa90429a843eb2cc8f7d9409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } } }
I20260923 22:12:34.442247 4037 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:34.442153 4038 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7527384aa90429a843eb2cc8f7d9409. Latest consensus state: current_term: 1 leader_uuid: "b7527384aa90429a843eb2cc8f7d9409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7527384aa90429a843eb2cc8f7d9409" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38871 } } }
I20260923 22:12:34.442339 4038 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:34.442454 4043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:34.442941 4043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:34.444805 4043 catalog_manager.cc:1347] Generated new cluster ID: 7044b86a5421454890729177139399b0
I20260923 22:12:34.444846 4043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:34.447854 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.436467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.1.132.190:38871)
0923 22:12:34.436648 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.436658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.436694 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:12:34.437236 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.437246 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.437314 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.437453 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.438213 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.439128 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.440211 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.440753 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.440843 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.443779 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.443783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.443783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:34.443796 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.445865 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.446333 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.446398 (+ 65us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.446399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.446627 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.446861 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.446862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.446863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.447198 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.447361 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.447484 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.447632 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260923 22:12:34.452346 4043 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:34.452893 4043 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:34.460767 4043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7527384aa90429a843eb2cc8f7d9409: Generated new TSK 0
I20260923 22:12:34.461023 4043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:34.492136 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:34.576018 4057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:34.576292 4057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:34.576329 4057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:34.577623 4057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:34.577668 4057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:34.577696 4057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:34.579224 4057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:34.579485 4057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:34.579712 4057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:34.584800 4065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.584877 4062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.584872 4063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.584903 4057 server_base.cc:1034] running on GCE node
I20260923 22:12:34.585261 4057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:34.585500 4057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:34.586663 4057 hybrid_clock.cc:648] HybridClock initialized: now 1790201554586645 us; error 30 us; skew 500 ppm
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:34.589298 4057 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:34.589864 4057 webserver.cc:460] Webserver started at http://127.1.132.129:43061/ using document root <none> and password file <none>
I20260923 22:12:34.590088 4057 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:34.590138 4057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:34.590266 4057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:34.591320 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "65619f97a48d4e1da96e619b80818b9d"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "87eb4169d9cd8bd1066c37bb976fb49a"
server_key_iv: "59233d3ba8265540132a53d791a68c39"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.591647 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "65619f97a48d4e1da96e619b80818b9d"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "87eb4169d9cd8bd1066c37bb976fb49a"
server_key_iv: "59233d3ba8265540132a53d791a68c39"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.593267 4057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:34.594507 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.594710 4057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:34.594803 4057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "65619f97a48d4e1da96e619b80818b9d"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "87eb4169d9cd8bd1066c37bb976fb49a"
server_key_iv: "59233d3ba8265540132a53d791a68c39"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.594872 4057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:34.622303 4057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:34.624845 4057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:34.624998 4057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:34.625234 4057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:34.625555 4057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:34.625595 4057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.625626 4057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:34.625643 4057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.631500 4057 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:43539
I20260923 22:12:34.631573 4185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:43539 every 8 connection(s)
I20260923 22:12:34.631918 4057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:34.637553 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4057
I20260923 22:12:34.637777 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:34.637952 1554 external_mini_cluster.cc:1428] Setting key adc16b43f3e7a1fb2c461d91bd459eb0
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:34.642244 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.632740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:54895 (local address 127.1.132.190:38871)
0923 22:12:34.632815 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.632818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.633046 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:12:34.633590 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.633591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.633602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.633660 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.634726 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.635246 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.636464 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.636689 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.636753 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.639197 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.639199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.639200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:34.639209 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.640801 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.641278 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.641280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.641280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.641339 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.641570 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.641571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.641571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.641707 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.641770 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.642051 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.642150 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:12:34.642421 4188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.632706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38871 (local address 127.1.132.129:54895)
0923 22:12:34.632898 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:34.632964 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:34.633437 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:34.633766 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:34.633770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:34.634374 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:34.634588 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.634593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.635367 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.635368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.636346 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.636349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.636812 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.636813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.636943 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.637446 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:34.637455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:34.639096 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:34.640894 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.640905 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.640915 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.641191 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.641396 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.641397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.641397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.641510 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.641761 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:34.641768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:34.641926 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 22:12:34.642020 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:12:34.642192 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260923 22:12:34.643100 4186 heartbeater.cc:347] Connected to a master server at 127.1.132.190:38871
I20260923 22:12:34.643216 4186 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:34.643445 4186 heartbeater.cc:511] Master 127.1.132.190:38871 requested a full tablet report, sending...
I20260923 22:12:34.644009 3995 ts_manager.cc:194] Registered new tserver with Master: 65619f97a48d4e1da96e619b80818b9d (127.1.132.129:43539)
I20260923 22:12:34.644721 3995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:54895
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:34.665146 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:34.744596 4191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:34.744807 4191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:34.744830 4191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:34.746017 4191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:34.746058 4191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:34.746086 4191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:34.747536 4191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:34.747761 4191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:34.747977 4191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:34.752820 4197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.752899 4196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.752903 4191 server_base.cc:1034] running on GCE node
W20260923 22:12:34.752820 4199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.753281 4191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:34.753515 4191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:34.754684 4191 hybrid_clock.cc:648] HybridClock initialized: now 1790201554754671 us; error 31 us; skew 500 ppm
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:34.757252 4191 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:34.757683 4191 webserver.cc:460] Webserver started at http://127.1.132.130:36521/ using document root <none> and password file <none>
I20260923 22:12:34.757887 4191 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:34.757943 4191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:34.758068 4191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:34.759177 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "0840e55431e3463d90d64dc6f7afcaa9"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "95a1359271535d4d52145a3fc9e02e3b"
server_key_iv: "399f9c5366cd35e859b1d6b0997503d5"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.759531 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "0840e55431e3463d90d64dc6f7afcaa9"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "95a1359271535d4d52145a3fc9e02e3b"
server_key_iv: "399f9c5366cd35e859b1d6b0997503d5"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.760875 4191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:34.761653 4206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.761816 4191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:34.761900 4191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "0840e55431e3463d90d64dc6f7afcaa9"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "95a1359271535d4d52145a3fc9e02e3b"
server_key_iv: "399f9c5366cd35e859b1d6b0997503d5"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.761961 4191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:34.781268 4191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:34.783529 4191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:34.783689 4191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:34.783934 4191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:34.784283 4191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:34.784327 4191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.784358 4191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:34.784373 4191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.789788 4191 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39175
I20260923 22:12:34.789834 4319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39175 every 8 connection(s)
I20260923 22:12:34.790194 4191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:34.799355 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.790891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:55387 (local address 127.1.132.190:38871)
0923 22:12:34.790963 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.790965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.791157 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:12:34.791670 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.791672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.791684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.791732 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.792744 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.793245 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.794227 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.794414 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.794468 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.796627 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.796629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.796629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:34.796638 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.798035 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.798417 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.798418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.798418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.798473 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.798695 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.798696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.798696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.798824 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.798884 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.799171 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.799260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:12:34.799546 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.790839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38871 (local address 127.1.132.130:55387)
0923 22:12:34.791037 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:34.791079 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:34.791569 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:34.791806 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:34.791810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:34.792446 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:34.792647 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.792653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.793330 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.793331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.794147 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.794149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.794484 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.794486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.794609 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.795111 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:34.795118 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:34.796526 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:34.798114 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.798122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.798125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.798349 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.798529 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.798530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.798530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.798639 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.798901 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:34.798907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:34.799050 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:12:34.799142 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:12:34.799307 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260923 22:12:34.800043 4320 heartbeater.cc:347] Connected to a master server at 127.1.132.190:38871
I20260923 22:12:34.800146 4320 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:34.800225 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4191
I20260923 22:12:34.800314 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:34.800369 4320 heartbeater.cc:511] Master 127.1.132.190:38871 requested a full tablet report, sending...
I20260923 22:12:34.800499 1554 external_mini_cluster.cc:1428] Setting key bf8b1fb85b797767783e7015e3ca0411
I20260923 22:12:34.800746 3995 ts_manager.cc:194] Registered new tserver with Master: 0840e55431e3463d90d64dc6f7afcaa9 (127.1.132.130:39175)
I20260923 22:12:34.801254 3995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:55387
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:34.827827 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:34.940366 4325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:34.940578 4325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:34.940609 4325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:34.941856 4325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:34.941898 4325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:34.941957 4325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:34.943660 4325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:38871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:34.943914 4325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:34.944139 4325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:34.949015 4333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.949045 4330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:34.949049 4331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:34.949203 4325 server_base.cc:1034] running on GCE node
I20260923 22:12:34.949560 4325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:34.949777 4325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:34.950912 4325 hybrid_clock.cc:648] HybridClock initialized: now 1790201554950898 us; error 30 us; skew 500 ppm
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:34.953339 4325 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:34.953800 4325 webserver.cc:460] Webserver started at http://127.1.132.131:33171/ using document root <none> and password file <none>
I20260923 22:12:34.954018 4325 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:34.954074 4325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:34.954197 4325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:34.955243 4325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "3a851bb06b68471bade083f5d941f191"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f95c43e3f8a57ad6befe04a9503102a"
server_key_iv: "c8d3cd569b58ef40741b9980308d55dd"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.955556 4325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "3a851bb06b68471bade083f5d941f191"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f95c43e3f8a57ad6befe04a9503102a"
server_key_iv: "c8d3cd569b58ef40741b9980308d55dd"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.956934 4325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:34.957739 4340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.957878 4325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:34.957965 4325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "3a851bb06b68471bade083f5d941f191"
format_stamp: "Formatted at 2026-09-23 22:12:34 on dist-test-slave-l4q5"
server_key: "4f95c43e3f8a57ad6befe04a9503102a"
server_key_iv: "c8d3cd569b58ef40741b9980308d55dd"
server_key_version: "encryptionkey@0"
I20260923 22:12:34.958027 4325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:34.973488 4325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:34.975629 4325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:34.975773 4325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:34.975992 4325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:34.976327 4325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:34.976364 4325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.976395 4325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:34.976419 4325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:34.982044 4325 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:32937
I20260923 22:12:34.982110 4453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:32937 every 8 connection(s)
I20260923 22:12:34.982415 4325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:34.983076 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4325
I20260923 22:12:34.983170 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:34.983366 1554 external_mini_cluster.cc:1428] Setting key 65bfee1415a07d8741c5ca60bf293a00
Sep 23 22:12:34 dist-test-slave-l4q5 krb5kdc[3950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:34.991797 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.983230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:37459 (local address 127.1.132.190:38871)
0923 22:12:34.983288 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:34.983290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:34.983700 (+ 410us) server_negotiation.cc:407] Connection header received
0923 22:12:34.984269 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:34.984270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:34.984281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:34.984320 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:34.985264 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.985786 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.986818 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.987024 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.987080 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.989159 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:34.989160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:34.989160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:34.989169 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:34.990454 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.990870 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.990872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.990872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.990928 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:34.991170 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:34.991171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:34.991172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:34.991289 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:34.991347 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:34.991619 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 22:12:34.991703 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:34.991967 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:34.983186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38871 (local address 127.1.132.131:37459)
0923 22:12:34.983578 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:34.983616 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:34.984136 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:34.984400 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:34.984405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:34.984973 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:34.985171 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.985176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.985873 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.985874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.986738 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:34.986741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:34.987095 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:34.987096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:34.987218 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:34.987700 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:34.987709 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:34.989068 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:34.990558 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.990565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.990568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.990801 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.990994 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:34.990995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:34.990995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:34.991111 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:34.991344 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:34.991350 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:34.991491 (+ 141us) client_negotiation.cc:769] Sending connection context
0923 22:12:34.991590 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:12:34.991755 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":80,"threads_started":1}
I20260923 22:12:34.992461 4454 heartbeater.cc:347] Connected to a master server at 127.1.132.190:38871
I20260923 22:12:34.992587 4454 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:34.992741 4454 heartbeater.cc:511] Master 127.1.132.190:38871 requested a full tablet report, sending...
I20260923 22:12:34.993101 3995 ts_manager.cc:194] Registered new tserver with Master: 3a851bb06b68471bade083f5d941f191 (127.1.132.131:32937)
I20260923 22:12:34.993441 3995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:37459
I20260923 22:12:34.995255 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:12:35.006687 4034 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:35.005692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.1.132.190:38871)
0923 22:12:35.005771 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:35.005773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:35.005811 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:12:35.005890 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:35.005891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:35.005900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:35.005945 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:35.006609 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41354: BlockingRecv error: recv got EOF from 127.0.0.1:41354 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:12:35.008078 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4057
I20260923 22:12:35.012758 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4191
I20260923 22:12:35.016621 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4325
I20260923 22:12:35.020668 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 3963
2026-09-23T22:12:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1826 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:35 dist-test-slave-l4q5 krb5kdc[4466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:35 dist-test-slave-l4q5 krb5kdc[4466](info): set up 2 sockets
Sep 23 22:12:35 dist-test-slave-l4q5 krb5kdc[4466](info): commencing operation
krb5kdc: starting...
W20260923 22:12:35.988270 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:36.107995 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40405
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:36.190758 4479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:36.191017 4479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:36.191053 4479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:36.192507 4479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:36.192566 4479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:36.192586 4479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:36.192605 4479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:36.192618 4479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:36.194299 4479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:36.194571 4479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:36.194785 4479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:36.199739 4484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:36.199851 4487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.199810 4479 server_base.cc:1034] running on GCE node
W20260923 22:12:36.199766 4485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:36.200243 4479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:36.200536 4479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:36.201713 4479 hybrid_clock.cc:648] HybridClock initialized: now 1790201556201697 us; error 31 us; skew 500 ppm
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:36.204392 4479 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:36.204929 4479 webserver.cc:460] Webserver started at http://127.1.132.190:40277/ using document root <none> and password file <none>
I20260923 22:12:36.205178 4479 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:36.205230 4479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:36.205356 4479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:36.206429 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-l4q5"
server_key: "c9f8b3fca4ff31254363a48a1ae84aae"
server_key_iv: "e6b72fc67a28150016e5f90684863d77"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.206763 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-l4q5"
server_key: "c9f8b3fca4ff31254363a48a1ae84aae"
server_key_iv: "e6b72fc67a28150016e5f90684863d77"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.208088 4479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:36.208892 4494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:36.209112 4479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:36.209213 4479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6"
format_stamp: "Formatted at 2026-09-23 22:12:36 on dist-test-slave-l4q5"
server_key: "c9f8b3fca4ff31254363a48a1ae84aae"
server_key_iv: "e6b72fc67a28150016e5f90684863d77"
server_key_version: "encryptionkey@0"
I20260923 22:12:36.209283 4479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:36.227661 4479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:36.229620 4479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:36.229760 4479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:36.233695 4479 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40405
I20260923 22:12:36.233740 4546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40405 every 8 connection(s)
I20260923 22:12:36.234200 4479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:36.234838 4547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:36.237182 4547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6: Bootstrap starting.
I20260923 22:12:36.237741 4547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:36.237988 4547 log.cc:824] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:36.238656 4547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6: No bootstrap required, opened a new log
I20260923 22:12:36.240365 4547 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } }
I20260923 22:12:36.240520 4547 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:36.240542 4547 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d23a5bbfe54a35b0ea26edf34c86b6, State: Initialized, Role: FOLLOWER
I20260923 22:12:36.240653 4547 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } }
I20260923 22:12:36.240725 4547 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:36.240756 4547 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:36.240789 4547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:36.241705 4547 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } }
I20260923 22:12:36.241791 4547 leader_election.cc:302] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d23a5bbfe54a35b0ea26edf34c86b6; no voters:
I20260923 22:12:36.241920 4547 leader_election.cc:288] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:36.242002 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:36.242147 4547 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:36.242157 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [term 1 LEADER]: Becoming Leader. State: Replica: d1d23a5bbfe54a35b0ea26edf34c86b6, State: Running, Role: LEADER
I20260923 22:12:36.242264 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } }
I20260923 22:12:36.242606 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d23a5bbfe54a35b0ea26edf34c86b6. Latest consensus state: current_term: 1 leader_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } } }
I20260923 22:12:36.242698 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:36.242926 4551 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d23a5bbfe54a35b0ea26edf34c86b6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40405 } } }
I20260923 22:12:36.242991 4551 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:36.243240 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4479
I20260923 22:12:36.243273 4559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:36.243332 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:36.243494 1554 external_mini_cluster.cc:1428] Setting key e3d299d68ed51b0f69498ea030c26084
I20260923 22:12:36.243752 4559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:36.245682 4559 catalog_manager.cc:1347] Generated new cluster ID: b9cafd2338384e009b17ce5fb87d796b
I20260923 22:12:36.245733 4559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:36.256223 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.244268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.1.132.190:40405)
0923 22:12:36.244446 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.244455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.244490 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:12:36.244693 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.244704 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.244775 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.244911 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.245631 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.246993 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.248059 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.248767 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.248881 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.252410 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.252415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.252416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:36.252431 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.254466 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.254931 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.254941 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.254941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.255054 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:36.255246 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:36.255246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:36.255247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:36.255553 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.255706 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.255805 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.255956 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260923 22:12:36.257529 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:36.259652 4559 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:36.260308 4559 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:36.267086 4559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1d23a5bbfe54a35b0ea26edf34c86b6: Generated new TSK 0
I20260923 22:12:36.267278 4559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:36.276911 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:36.273008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.1.132.190:40405)
0923 22:12:36.273073 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:36.273075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:36.273167 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:12:36.273239 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:36.273240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:36.273249 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:36.273298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:36.273739 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.274273 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.275341 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:36.275553 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:36.275606 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:36.276323 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:36.276325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:36.276325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:36.276573 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:36.276591 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:36.276598 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:36.276649 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:36.276765 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 22:12:36.276822 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:36.279872 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4479
2026-09-23T22:12:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1257 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4580](info): set up 2 sockets
Sep 23 22:12:36 dist-test-slave-l4q5 krb5kdc[4580](info): commencing operation
krb5kdc: starting...
W20260923 22:12:37.243793 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:37.359778 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39305
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.442711 4593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.442937 4593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.442970 4593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.444172 4593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:37.444255 4593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.444276 4593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.444293 4593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:37.444312 4593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:37.445868 4593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:37.446132 4593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.446344 4593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:37.451270 4599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.451297 4601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.451359 4593 server_base.cc:1034] running on GCE node
W20260923 22:12:37.451270 4598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.451735 4593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.452078 4593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.453248 4593 hybrid_clock.cc:648] HybridClock initialized: now 1790201557453236 us; error 48 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.455818 4593 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:37.456455 4593 webserver.cc:460] Webserver started at http://127.1.132.190:38471/ using document root <none> and password file <none>
I20260923 22:12:37.456728 4593 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.456791 4593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.457003 4593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.458137 4593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "f3375614abe845dca0e128851677e870"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "0a4e0382fad431211f0c3c3cf7f489ba"
server_key_iv: "5a4c6eaca831996ccc0d15a4ad623d72"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.458503 4593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "f3375614abe845dca0e128851677e870"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "0a4e0382fad431211f0c3c3cf7f489ba"
server_key_iv: "5a4c6eaca831996ccc0d15a4ad623d72"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.459980 4593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:37.460896 4608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.461124 4593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:37.461218 4593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "f3375614abe845dca0e128851677e870"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "0a4e0382fad431211f0c3c3cf7f489ba"
server_key_iv: "5a4c6eaca831996ccc0d15a4ad623d72"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.461318 4593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.478145 4593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:37.480734 4593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:37.480865 4593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:37.484666 4593 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:39305
I20260923 22:12:37.484807 4660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:39305 every 8 connection(s)
I20260923 22:12:37.485044 4593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:37.485880 4661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:37.488129 4661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870: Bootstrap starting.
I20260923 22:12:37.488701 4661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:37.488955 4661 log.cc:824] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:37.489607 4661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870: No bootstrap required, opened a new log
I20260923 22:12:37.491186 4661 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } }
I20260923 22:12:37.491333 4661 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:37.491355 4661 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3375614abe845dca0e128851677e870, State: Initialized, Role: FOLLOWER
I20260923 22:12:37.491505 4661 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } }
I20260923 22:12:37.491583 4661 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:37.491607 4661 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:37.491654 4661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:37.492566 4661 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } }
I20260923 22:12:37.492656 4661 leader_election.cc:302] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3375614abe845dca0e128851677e870; no voters:
I20260923 22:12:37.492808 4661 leader_election.cc:288] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:37.492846 4664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:37.492993 4664 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [term 1 LEADER]: Becoming Leader. State: Replica: f3375614abe845dca0e128851677e870, State: Running, Role: LEADER
I20260923 22:12:37.493070 4661 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:37.493083 4664 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } }
I20260923 22:12:37.493443 4665 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3375614abe845dca0e128851677e870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } } }
I20260923 22:12:37.493454 4666 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3375614abe845dca0e128851677e870. Latest consensus state: current_term: 1 leader_uuid: "f3375614abe845dca0e128851677e870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3375614abe845dca0e128851677e870" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39305 } } }
I20260923 22:12:37.493569 4666 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:37.493765 4672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:37.494293 4672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:37.493755 4665 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:37.494684 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4593
I20260923 22:12:37.494784 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:37.494951 1554 external_mini_cluster.cc:1428] Setting key 206429a8d0fe1b0b35261616dddea390
I20260923 22:12:37.496009 4672 catalog_manager.cc:1347] Generated new cluster ID: ccf512c49db1453782902f9767b5743d
I20260923 22:12:37.496057 4672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:37.504350 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.495559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.1.132.190:39305)
0923 22:12:37.495699 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:37.495709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:37.495736 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:37.495832 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:37.495837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:37.495881 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:37.496048 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:37.496725 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.497674 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.498413 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.498851 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.498942 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.501133 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:37.501138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:37.501138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:37.501150 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:37.502688 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.503013 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.503016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.503017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.503124 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.503324 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.503324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.503325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.503631 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:37.503782 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:37.503892 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:12:37.504134 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260923 22:12:37.507146 4672 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:37.507647 4672 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:37.513885 4672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3375614abe845dca0e128851677e870: Generated new TSK 0
I20260923 22:12:37.514091 4672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:37.547070 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.646895 4687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.647119 4687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.647145 4687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.648401 4687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.648444 4687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.648471 4687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:37.650024 4687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:37.650259 4687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.650465 4687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:37.655653 4692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.655658 4693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.655673 4687 server_base.cc:1034] running on GCE node
W20260923 22:12:37.655653 4695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.656101 4687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.656358 4687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.657512 4687 hybrid_clock.cc:648] HybridClock initialized: now 1790201557657497 us; error 28 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.660010 4687 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:37.660498 4687 webserver.cc:460] Webserver started at http://127.1.132.129:34863/ using document root <none> and password file <none>
I20260923 22:12:37.660708 4687 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.660758 4687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.660879 4687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.661967 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "fba019ec86104952aa9a1f6f7537a15c"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "a59e567358680eca07e2cfb9ea971e8f"
server_key_iv: "c3c2d6790c3181ab716bea6353be9ff7"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.662305 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "fba019ec86104952aa9a1f6f7537a15c"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "a59e567358680eca07e2cfb9ea971e8f"
server_key_iv: "c3c2d6790c3181ab716bea6353be9ff7"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.663887 4687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:37.664865 4702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.665040 4687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:37.665207 4687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "fba019ec86104952aa9a1f6f7537a15c"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "a59e567358680eca07e2cfb9ea971e8f"
server_key_iv: "c3c2d6790c3181ab716bea6353be9ff7"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.665295 4687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.689432 4687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:37.691702 4687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:37.691864 4687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:37.692103 4687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:37.692471 4687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:37.692508 4687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.692538 4687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:37.692560 4687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.697979 4687 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:45079
I20260923 22:12:37.698015 4815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:45079 every 8 connection(s)
I20260923 22:12:37.698335 4687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:37.702659 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4687
I20260923 22:12:37.702757 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:37.702930 1554 external_mini_cluster.cc:1428] Setting key 8fb47c59724224e02dc8e593c0bd34a5
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:37.708014 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.699059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:47139 (local address 127.1.132.190:39305)
0923 22:12:37.699143 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:37.699145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:37.699304 (+ 159us) server_negotiation.cc:407] Connection header received
0923 22:12:37.699855 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:37.699856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:37.699868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:37.699917 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:37.700878 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.701386 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.702458 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.702692 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.702764 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.705252 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:37.705253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:37.705254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:37.705260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:37.706579 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.706976 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.706977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.706978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.707034 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.707295 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.707296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.707297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.707426 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:37.707487 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:37.707861 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 22:12:37.707944 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:12:37.708309 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.699027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39305 (local address 127.1.132.129:47139)
0923 22:12:37.699177 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:37.699223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:37.699767 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:37.699976 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:37.699979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:37.700573 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:37.700771 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.700776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.701489 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.701490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.702340 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.702343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.702803 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.702804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.702942 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.703515 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:37.703526 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:37.705157 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:37.706664 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.706672 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.706675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.706906 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.707094 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.707095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.707095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.707217 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.707495 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:37.707502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:37.707688 (+ 186us) client_negotiation.cc:769] Sending connection context
0923 22:12:37.707818 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 22:12:37.708004 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260923 22:12:37.708839 4816 heartbeater.cc:347] Connected to a master server at 127.1.132.190:39305
I20260923 22:12:37.708935 4816 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:37.709110 4816 heartbeater.cc:511] Master 127.1.132.190:39305 requested a full tablet report, sending...
I20260923 22:12:37.709684 4625 ts_manager.cc:194] Registered new tserver with Master: fba019ec86104952aa9a1f6f7537a15c (127.1.132.129:45079)
I20260923 22:12:37.710314 4625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:47139
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:37.729606 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.809247 4821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.809473 4821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.809495 4821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.810683 4821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.810729 4821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.810755 4821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:37.812265 4821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:37.812505 4821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.812734 4821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:37.817821 4821 server_base.cc:1034] running on GCE node
W20260923 22:12:37.817780 4827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.817916 4826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.817780 4829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.818176 4821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.818377 4821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.819525 4821 hybrid_clock.cc:648] HybridClock initialized: now 1790201557819508 us; error 30 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.822039 4821 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:37.822476 4821 webserver.cc:460] Webserver started at http://127.1.132.130:45493/ using document root <none> and password file <none>
I20260923 22:12:37.822657 4821 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.822695 4821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.822788 4821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.823789 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "75f57697daf4439a8b6688d9254b7c4d"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "fcf38fa177a7824e4366e79625bb8965"
server_key_iv: "1b0d823eac78377250a9758b7ab32908"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.824093 4821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "75f57697daf4439a8b6688d9254b7c4d"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "fcf38fa177a7824e4366e79625bb8965"
server_key_iv: "1b0d823eac78377250a9758b7ab32908"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.825399 4821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:37.826136 4836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.826278 4821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:37.826362 4821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "75f57697daf4439a8b6688d9254b7c4d"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "fcf38fa177a7824e4366e79625bb8965"
server_key_iv: "1b0d823eac78377250a9758b7ab32908"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.826480 4821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.842568 4821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:37.844384 4821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:37.844525 4821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:37.844750 4821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:37.845045 4821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:37.845077 4821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.845108 4821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:37.845131 4821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.850540 4821 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:34141
I20260923 22:12:37.850584 4949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:34141 every 8 connection(s)
I20260923 22:12:37.850884 4821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:37.854383 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4821
I20260923 22:12:37.854496 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:37.854660 1554 external_mini_cluster.cc:1428] Setting key d6d9a58b5d8da864694ccdbc0f91a34f
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:37.860651 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.851623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:43809 (local address 127.1.132.190:39305)
0923 22:12:37.851692 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:37.851694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:37.851885 (+ 191us) server_negotiation.cc:407] Connection header received
0923 22:12:37.852446 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:37.852448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:37.852519 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:37.852569 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:37.853488 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.853989 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.855046 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.855237 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.855291 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.857361 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:37.857363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:37.857363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:37.857371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:37.858955 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.859412 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.859413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.859413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.859508 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:37.859851 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:37.859853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:37.859853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:37.860017 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:37.860113 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:37.860441 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 22:12:37.860540 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:12:37.860733 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:37.851586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39305 (local address 127.1.132.130:43809)
0923 22:12:37.851774 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:37.851810 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:37.852353 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:37.852649 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:37.852653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:37.853202 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:37.853394 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.853399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.854068 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.854069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.854973 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:37.854975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:37.855301 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:37.855302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:37.855434 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:37.855899 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:37.855907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:37.857270 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:37.859063 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.859070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.859072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.859316 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.859639 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:37.859640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:37.859641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:37.859758 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:37.860111 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:37.860118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:37.860300 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 22:12:37.860404 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:12:37.860552 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260923 22:12:37.861277 4950 heartbeater.cc:347] Connected to a master server at 127.1.132.190:39305
I20260923 22:12:37.861373 4950 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:37.861557 4950 heartbeater.cc:511] Master 127.1.132.190:39305 requested a full tablet report, sending...
I20260923 22:12:37.861956 4625 ts_manager.cc:194] Registered new tserver with Master: 75f57697daf4439a8b6688d9254b7c4d (127.1.132.130:34141)
I20260923 22:12:37.862455 4625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:43809
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:37.879274 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:37.959304 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:37.959553 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:37.959589 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:37.960915 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:37.960968 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:37.961009 4955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:37.962584 4955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:39305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:37.962822 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:37.963035 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:37.968238 4960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.968256 4961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:37.968254 4963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:37.968369 4955 server_base.cc:1034] running on GCE node
I20260923 22:12:37.968657 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:37.968894 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:37.970031 4955 hybrid_clock.cc:648] HybridClock initialized: now 1790201557970020 us; error 28 us; skew 500 ppm
Sep 23 22:12:37 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:37.972476 4955 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:37.972908 4955 webserver.cc:460] Webserver started at http://127.1.132.131:34303/ using document root <none> and password file <none>
I20260923 22:12:37.973098 4955 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:37.973152 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:37.973275 4955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:37.974324 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "3b9a7f6c079945ef9578d13f8e1cc490"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "3cc4d869efdfd55ffaeba61ca61d1256"
server_key_iv: "07b20fe49bc2e49a8b933b12e86b16cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.974665 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "3b9a7f6c079945ef9578d13f8e1cc490"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "3cc4d869efdfd55ffaeba61ca61d1256"
server_key_iv: "07b20fe49bc2e49a8b933b12e86b16cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.975940 4955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:37.976770 4970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:37.976943 4955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:37.977023 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "3b9a7f6c079945ef9578d13f8e1cc490"
format_stamp: "Formatted at 2026-09-23 22:12:37 on dist-test-slave-l4q5"
server_key: "3cc4d869efdfd55ffaeba61ca61d1256"
server_key_iv: "07b20fe49bc2e49a8b933b12e86b16cb"
server_key_version: "encryptionkey@0"
I20260923 22:12:37.977090 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:37.998453 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:38.000597 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:38.000739 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:38.000962 4955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:38.001251 4955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:38.001283 4955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.001312 4955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:38.001335 4955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:38.006835 4955 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:37339
I20260923 22:12:38.006883 5083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:37339 every 8 connection(s)
I20260923 22:12:38.007189 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[4580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:38.014012 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 4955
I20260923 22:12:38.014091 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:38.014256 1554 external_mini_cluster.cc:1428] Setting key 16eef243c5f5ff75d0c18c368c37387c
I20260923 22:12:38.016590 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.007828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:45383 (local address 127.1.132.190:39305)
0923 22:12:38.007902 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.007904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.008083 (+ 179us) server_negotiation.cc:407] Connection header received
0923 22:12:38.008642 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.008643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.008654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.008730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.009646 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.010178 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.011179 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.011370 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.011429 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.013593 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.013595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.013595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.013604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.015322 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.015684 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.015685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.015685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.015744 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.015960 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.015961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.015961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.016075 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.016137 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.016418 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.016503 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:12:38.016772 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.007796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39305 (local address 127.1.132.131:45383)
0923 22:12:38.007964 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:38.008002 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:38.008526 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:38.008785 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:38.008788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:38.009351 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:38.009557 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.009562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.010258 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.010259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.011092 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:38.011094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.011467 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.011468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:38.011583 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.012027 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:38.012035 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:38.013483 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:38.015391 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.015398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.015401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.015612 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.015795 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:38.015795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:38.015796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:38.015904 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:38.016127 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:38.016133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:38.016300 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 22:12:38.016390 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:12:38.016549 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260923 22:12:38.017194 5084 heartbeater.cc:347] Connected to a master server at 127.1.132.190:39305
I20260923 22:12:38.017288 5084 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:38.017464 5084 heartbeater.cc:511] Master 127.1.132.190:39305 requested a full tablet report, sending...
I20260923 22:12:38.017783 4625 ts_manager.cc:194] Registered new tserver with Master: 3b9a7f6c079945ef9578d13f8e1cc490 (127.1.132.131:37339)
I20260923 22:12:38.018126 4625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:45383
I20260923 22:12:38.025849 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[4580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[4580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201558, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:38.047389 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:38.040447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.1.132.190:39305)
0923 22:12:38.040521 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:38.040524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:38.040637 (+ 113us) server_negotiation.cc:407] Connection header received
0923 22:12:38.040687 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:38.040688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:38.040697 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:38.040744 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:38.041255 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.041753 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.042438 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:38.042641 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:38.042690 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:38.044849 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:38.044850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:38.044851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:38.044858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:38.046276 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.046607 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.046608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.046608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.046669 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:38.046872 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:38.046873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:38.046873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:38.047047 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:38.047143 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:38.047253 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:12:38.047319 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260923 22:12:38.047639 4625 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32862
I20260923 22:12:38.048125 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4687
I20260923 22:12:38.052794 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4821
I20260923 22:12:38.056589 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4955
I20260923 22:12:38.060171 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 4593
2026-09-23T22:12:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1781 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[5096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[5096](info): set up 2 sockets
Sep 23 22:12:38 dist-test-slave-l4q5 krb5kdc[5096](info): commencing operation
krb5kdc: starting...
W20260923 22:12:39.015913 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:39.136257 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36715
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.220594 5109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.220817 5109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.220846 5109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.222048 5109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:39.222095 5109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.222113 5109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.222136 5109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:39.222169 5109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:39.223748 5109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:39.223991 5109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.224244 5109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.229064 5115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.229127 5109 server_base.cc:1034] running on GCE node
W20260923 22:12:39.229128 5114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.229067 5117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.229526 5109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.229805 5109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.230943 5109 hybrid_clock.cc:648] HybridClock initialized: now 1790201559230934 us; error 32 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.245688 5109 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:39.246166 5109 webserver.cc:460] Webserver started at http://127.1.132.190:44769/ using document root <none> and password file <none>
I20260923 22:12:39.246389 5109 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.246438 5109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.246564 5109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.247604 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "8146df14c84c40e7808dab58b302a849"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "982331c641867042b9e547673cc23f00"
server_key_iv: "8382fdb4424b60991eede090c217118e"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.247938 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "8146df14c84c40e7808dab58b302a849"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "982331c641867042b9e547673cc23f00"
server_key_iv: "8382fdb4424b60991eede090c217118e"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.249253 5109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:39.249966 5124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.250139 5109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:39.250221 5109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "8146df14c84c40e7808dab58b302a849"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "982331c641867042b9e547673cc23f00"
server_key_iv: "8382fdb4424b60991eede090c217118e"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.250288 5109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.260522 5109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.262491 5109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.262633 5109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.266978 5109 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36715
I20260923 22:12:39.267021 5176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36715 every 8 connection(s)
I20260923 22:12:39.267501 5109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:39.268203 5177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:39.270467 5177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849: Bootstrap starting.
I20260923 22:12:39.271134 5177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:39.271373 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5109
I20260923 22:12:39.271407 5177 log.cc:824] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:39.271472 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:39.271623 1554 external_mini_cluster.cc:1428] Setting key b2091bec6bac5a6893cf6d4d16e8152a
I20260923 22:12:39.272119 5177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849: No bootstrap required, opened a new log
I20260923 22:12:39.274295 5177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } }
I20260923 22:12:39.274447 5177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:39.274469 5177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8146df14c84c40e7808dab58b302a849, State: Initialized, Role: FOLLOWER
I20260923 22:12:39.274606 5177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } }
I20260923 22:12:39.274685 5177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:39.274725 5177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:39.274765 5177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:39.275449 5177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } }
I20260923 22:12:39.275563 5177 leader_election.cc:302] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8146df14c84c40e7808dab58b302a849; no voters:
I20260923 22:12:39.275743 5177 leader_election.cc:288] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:39.275789 5182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:39.275897 5182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [term 1 LEADER]: Becoming Leader. State: Replica: 8146df14c84c40e7808dab58b302a849, State: Running, Role: LEADER
I20260923 22:12:39.275979 5182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } }
I20260923 22:12:39.276018 5177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:39.276484 5184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8146df14c84c40e7808dab58b302a849. Latest consensus state: current_term: 1 leader_uuid: "8146df14c84c40e7808dab58b302a849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } } }
I20260923 22:12:39.276578 5184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:39.276749 5183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8146df14c84c40e7808dab58b302a849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8146df14c84c40e7808dab58b302a849" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36715 } } }
I20260923 22:12:39.276801 5183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:39.277073 5188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:39.277576 5188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:39.279170 5188 catalog_manager.cc:1347] Generated new cluster ID: bce6b8ba95a649798cab6b3457e0a267
I20260923 22:12:39.279220 5188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:39.284073 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.272122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.1.132.190:36715)
0923 22:12:39.272337 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.272345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.272371 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:39.274290 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.274299 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.274371 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.274585 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.275284 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.276545 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.277636 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.278087 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.278185 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.280875 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.280880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.280881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.280892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.282464 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.282808 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.282812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.282812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.282913 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.283089 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.283089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.283090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.283391 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.283558 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.283693 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.283837 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260923 22:12:39.308712 5188 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:39.309307 5188 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:39.311625 5188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8146df14c84c40e7808dab58b302a849: Generated new TSK 0
I20260923 22:12:39.311803 5188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:39.348827 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.444949 5203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.445163 5203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.445189 5203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.446367 5203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.446411 5203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.446440 5203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:39.447978 5203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:39.448205 5203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.448398 5203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.453013 5208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.453100 5209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.453162 5211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.453476 5203 server_base.cc:1034] running on GCE node
I20260923 22:12:39.453691 5203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.453917 5203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.455049 5203 hybrid_clock.cc:648] HybridClock initialized: now 1790201559455021 us; error 45 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.457515 5203 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:39.457955 5203 webserver.cc:460] Webserver started at http://127.1.132.129:33411/ using document root <none> and password file <none>
I20260923 22:12:39.458175 5203 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.458223 5203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.458346 5203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.459390 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "41dc0addd93d4203a0f7afe6fe2abc3c"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "8c89cabff4bb43b77d9d08472b5073fc"
server_key_iv: "ca7981aba2f417ce5e3e6f87fd7052d2"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.459740 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "41dc0addd93d4203a0f7afe6fe2abc3c"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "8c89cabff4bb43b77d9d08472b5073fc"
server_key_iv: "ca7981aba2f417ce5e3e6f87fd7052d2"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.461261 5203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:39.462203 5218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.462416 5203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:39.462513 5203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "41dc0addd93d4203a0f7afe6fe2abc3c"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "8c89cabff4bb43b77d9d08472b5073fc"
server_key_iv: "ca7981aba2f417ce5e3e6f87fd7052d2"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.462579 5203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.483736 5203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.485968 5203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.486104 5203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.486375 5203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:39.486677 5203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:39.486711 5203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.486739 5203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:39.486763 5203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.492374 5203 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:38073
I20260923 22:12:39.492429 5331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:38073 every 8 connection(s)
I20260923 22:12:39.492753 5203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:39.494578 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5203
I20260923 22:12:39.494678 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:39.494848 1554 external_mini_cluster.cc:1428] Setting key a6a3e095de91699d57b7226d017a59d6
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:12:39.503296 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.493476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:43921 (local address 127.1.132.190:36715)
0923 22:12:39.493519 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.493521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.493705 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:39.494720 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.494721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.494734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.494804 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.495622 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.496155 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.497354 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.497649 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.497723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.500438 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.500441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.500441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.500451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.501920 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.502349 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.502351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.502352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.502413 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.502621 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.502622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.502622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.502767 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.502834 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.503144 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.503229 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:39.503450 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.493454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36715 (local address 127.1.132.129:43921)
0923 22:12:39.493598 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:39.493635 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:39.494576 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:39.494880 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:39.494884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:39.495238 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:39.495477 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.495484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.496263 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.496263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.497250 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.497252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.497750 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.497751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.497879 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.498516 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:39.498525 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:39.500330 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:39.502024 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.502031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.502034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.502274 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.502473 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.502474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.502474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.502566 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.502845 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:39.502851 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:39.503015 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 22:12:39.503105 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:12:39.503241 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260923 22:12:39.504078 5332 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36715
I20260923 22:12:39.504173 5332 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:39.504369 5332 heartbeater.cc:511] Master 127.1.132.190:36715 requested a full tablet report, sending...
I20260923 22:12:39.504901 5141 ts_manager.cc:194] Registered new tserver with Master: 41dc0addd93d4203a0f7afe6fe2abc3c (127.1.132.129:38073)
I20260923 22:12:39.505627 5141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:43921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:39.523118 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.605569 5337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.605803 5337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.605839 5337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.607067 5337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.607120 5337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.607158 5337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:39.608789 5337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:39.609054 5337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.609277 5337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.614300 5345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.614308 5343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.614379 5337 server_base.cc:1034] running on GCE node
W20260923 22:12:39.614308 5342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.614722 5337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.614949 5337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.616091 5337 hybrid_clock.cc:648] HybridClock initialized: now 1790201559616078 us; error 30 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.618610 5337 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:39.619036 5337 webserver.cc:460] Webserver started at http://127.1.132.130:39277/ using document root <none> and password file <none>
I20260923 22:12:39.619259 5337 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.619309 5337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.619443 5337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.620540 5337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "e3feca5f8eec4e288a70023660cef668"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "395e2c779e859f1dde381a44dec8ee5b"
server_key_iv: "c27715206e58b81b4f7f78eced0c72c7"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.620862 5337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "e3feca5f8eec4e288a70023660cef668"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "395e2c779e859f1dde381a44dec8ee5b"
server_key_iv: "c27715206e58b81b4f7f78eced0c72c7"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.622107 5337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:39.622808 5352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.622978 5337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:39.623060 5337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "e3feca5f8eec4e288a70023660cef668"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "395e2c779e859f1dde381a44dec8ee5b"
server_key_iv: "c27715206e58b81b4f7f78eced0c72c7"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.623122 5337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.647007 5337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.649199 5337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.649339 5337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.649565 5337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:39.649921 5337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:39.649974 5337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.650005 5337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:39.650030 5337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.656531 5337 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:45779
I20260923 22:12:39.656590 5465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:45779 every 8 connection(s)
I20260923 22:12:39.656908 5337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:39.658197 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5337
I20260923 22:12:39.658300 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:39.658480 1554 external_mini_cluster.cc:1428] Setting key 1374065db4afb537f412306ef4e2c471
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:12:39.667456 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.657626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:33647 (local address 127.1.132.190:36715)
0923 22:12:39.657713 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.657715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.657883 (+ 168us) server_negotiation.cc:407] Connection header received
0923 22:12:39.659090 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.659091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.659102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.659166 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.660050 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.660641 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.661880 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.662084 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.662145 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.664565 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.664567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.664567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.664574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.665985 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.666478 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.666479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.666479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.666534 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.666822 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.666823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.666823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.666938 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.666990 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.667282 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.667351 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:12:39.667654 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.657617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36715 (local address 127.1.132.130:33647)
0923 22:12:39.657766 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:39.657804 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:39.658957 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:39.659246 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:39.659250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:39.659621 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:39.659917 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.659923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.660757 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.660758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.661771 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.661774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.662164 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.662165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.662351 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.662890 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:39.662900 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:39.664430 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:39.666073 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.666082 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.666084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.666373 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.666645 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.666647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.666647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.666763 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.667023 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:39.667028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:39.667166 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 22:12:39.667268 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:12:39.667403 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260923 22:12:39.668251 5466 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36715
I20260923 22:12:39.668339 5466 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:39.668498 5466 heartbeater.cc:511] Master 127.1.132.190:36715 requested a full tablet report, sending...
I20260923 22:12:39.668895 5141 ts_manager.cc:194] Registered new tserver with Master: e3feca5f8eec4e288a70023660cef668 (127.1.132.130:45779)
I20260923 22:12:39.669274 5141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:33647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:39.688408 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:39.768711 5471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:39.768919 5471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:39.768944 5471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:39.770138 5471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:39.770184 5471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:39.770210 5471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:39.771799 5471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:39.772029 5471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:39.772275 5471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:39.777098 5477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.777223 5471 server_base.cc:1034] running on GCE node
W20260923 22:12:39.777112 5476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:39.777307 5479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:39.777529 5471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:39.777765 5471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:39.778926 5471 hybrid_clock.cc:648] HybridClock initialized: now 1790201559778914 us; error 29 us; skew 500 ppm
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:39.781692 5471 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:39.782135 5471 webserver.cc:460] Webserver started at http://127.1.132.131:40723/ using document root <none> and password file <none>
I20260923 22:12:39.782351 5471 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:39.782400 5471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:39.782532 5471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:39.783597 5471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "1e7aecd8343543cd8fa00666c12b4e59"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "e263502a98615a1c1b4fddd840687e1d"
server_key_iv: "9a3557228ffaae21eaae39fef4c98be0"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.783921 5471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "1e7aecd8343543cd8fa00666c12b4e59"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "e263502a98615a1c1b4fddd840687e1d"
server_key_iv: "9a3557228ffaae21eaae39fef4c98be0"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.785439 5471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:39.786376 5486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.786556 5471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:39.786648 5471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "1e7aecd8343543cd8fa00666c12b4e59"
format_stamp: "Formatted at 2026-09-23 22:12:39 on dist-test-slave-l4q5"
server_key: "e263502a98615a1c1b4fddd840687e1d"
server_key_iv: "9a3557228ffaae21eaae39fef4c98be0"
server_key_version: "encryptionkey@0"
I20260923 22:12:39.786722 5471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:39.811846 5471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:39.813793 5471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:39.813910 5471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:39.814075 5471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:39.814353 5471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:39.814384 5471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.814414 5471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:39.814437 5471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:39.819718 5471 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:34873
I20260923 22:12:39.819814 5599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:34873 every 8 connection(s)
I20260923 22:12:39.820061 5471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:39.823545 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5471
I20260923 22:12:39.823637 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:39.823788 1554 external_mini_cluster.cc:1428] Setting key c8497a00b24b70363165f7f26a425437
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:39.829028 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.820762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:42785 (local address 127.1.132.190:36715)
0923 22:12:39.820825 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.820827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.821008 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:12:39.821925 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.821926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.821939 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.821998 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.822629 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.823127 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.824177 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.824456 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.824515 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.826571 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.826572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.826572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.826580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.827794 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.828151 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.828153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.828153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.828235 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.828444 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.828445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.828445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.828555 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.828610 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.828858 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.828931 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:39.829191 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.820741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36715 (local address 127.1.132.131:42785)
0923 22:12:39.820897 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:39.820931 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:39.821826 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:39.822065 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:39.822068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:39.822317 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:39.822540 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.822544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.823208 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.823209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.824106 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:39.824108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.824536 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.824537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:39.824665 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.825142 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:39.825150 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:39.826486 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:39.827866 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.827872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.827874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.828089 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.828290 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:39.828290 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:39.828291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:39.828390 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:39.828609 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:39.828615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:39.828745 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 22:12:39.828832 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:12:39.828981 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260923 22:12:39.829793 5600 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36715
I20260923 22:12:39.829882 5600 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:39.830025 5600 heartbeater.cc:511] Master 127.1.132.190:36715 requested a full tablet report, sending...
I20260923 22:12:39.830353 5141 ts_manager.cc:194] Registered new tserver with Master: 1e7aecd8343543cd8fa00666c12b4e59 (127.1.132.131:34873)
I20260923 22:12:39.830717 5141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:42785
I20260923 22:12:39.835383 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:39.857823 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.849799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.1.132.129:38073)
0923 22:12:39.849948 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.849952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.849963 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:39.849992 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.849994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.850008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.850077 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.850623 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.851368 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.852170 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.852596 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.852658 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.855224 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.855237 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.855237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.855245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.856738 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.857064 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.857067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.857067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.857119 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.857278 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.857279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.857279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.857418 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.857565 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.857636 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.857717 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:39.866022 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:39.859180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.1.132.190:36715)
0923 22:12:39.859244 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:39.859246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:39.859255 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:39.859281 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:39.859282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:39.859292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:39.859337 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:39.859788 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.860346 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.860990 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:39.861187 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:39.861239 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:39.863525 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:39.863527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:39.863527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:39.863534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:39.864766 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.865160 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.865162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.865162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.865217 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:39.865440 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:39.865441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:39.865442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:39.865641 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:39.865725 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:39.865849 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 22:12:39.865911 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260923 22:12:39.866230 5141 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37110
I20260923 22:12:39.866564 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5203
I20260923 22:12:39.871295 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5337
I20260923 22:12:39.875451 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5471
I20260923 22:12:39.879493 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5109
2026-09-23T22:12:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1819 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5612](info): set up 2 sockets
Sep 23 22:12:39 dist-test-slave-l4q5 krb5kdc[5612](info): commencing operation
krb5kdc: starting...
W20260923 22:12:40.831947 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:40 dist-test-slave-l4q5 krb5kdc[5612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:40.951411 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35191
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:41.031594 5625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.031826 5625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.031849 5625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.033114 5625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:41.033166 5625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.033180 5625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.033193 5625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:41.033217 5625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:41.034785 5625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:41.035027 5625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.035247 5625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.039785 5630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.039786 5631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.039970 5633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.040035 5625 server_base.cc:1034] running on GCE node
I20260923 22:12:41.040293 5625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.040570 5625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.041707 5625 hybrid_clock.cc:648] HybridClock initialized: now 1790201561041694 us; error 29 us; skew 500 ppm
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:41.044238 5625 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:41.044584 5625 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.044646 5625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.044778 5625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.045892 5625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "711ead9275c14858acbda2c1922150be"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d4d999080a0e739fb2f4a84bb9690a04"
server_key_iv: "115b21282ad1a0b49d5a49100f8a8fbc"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.046233 5625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "711ead9275c14858acbda2c1922150be"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d4d999080a0e739fb2f4a84bb9690a04"
server_key_iv: "115b21282ad1a0b49d5a49100f8a8fbc"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.047526 5625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:41.048362 5639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.048564 5625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:41.048669 5625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "711ead9275c14858acbda2c1922150be"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d4d999080a0e739fb2f4a84bb9690a04"
server_key_iv: "115b21282ad1a0b49d5a49100f8a8fbc"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.048744 5625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.066200 5625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.068120 5625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.068280 5625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.072063 5625 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35191
I20260923 22:12:41.072093 5691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35191 every 8 connection(s)
I20260923 22:12:41.072441 5625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:41.073391 5692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.075682 5692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: Bootstrap starting.
I20260923 22:12:41.076129 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5625
I20260923 22:12:41.076246 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:41.076404 5692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.076436 1554 external_mini_cluster.cc:1428] Setting key fef3b322202459b598de82619343202e
I20260923 22:12:41.076712 5692 log.cc:824] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.077461 5692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: No bootstrap required, opened a new log
I20260923 22:12:41.079609 5692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } }
I20260923 22:12:41.080246 5692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.080293 5692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711ead9275c14858acbda2c1922150be, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.080435 5692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } }
I20260923 22:12:41.080523 5692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:41.080569 5692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:41.080626 5692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.081539 5692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } }
I20260923 22:12:41.081647 5692 leader_election.cc:302] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 711ead9275c14858acbda2c1922150be; no voters:
I20260923 22:12:41.081835 5692 leader_election.cc:288] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:41.081885 5697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.082000 5697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [term 1 LEADER]: Becoming Leader. State: Replica: 711ead9275c14858acbda2c1922150be, State: Running, Role: LEADER
I20260923 22:12:41.082082 5697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } }
I20260923 22:12:41.082135 5692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:41.082645 5698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "711ead9275c14858acbda2c1922150be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } } }
I20260923 22:12:41.082727 5698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.083130 5699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 711ead9275c14858acbda2c1922150be. Latest consensus state: current_term: 1 leader_uuid: "711ead9275c14858acbda2c1922150be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711ead9275c14858acbda2c1922150be" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35191 } } }
I20260923 22:12:41.083278 5699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:41.083406 5706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:41.084118 5706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:41.086014 5706 catalog_manager.cc:1347] Generated new cluster ID: 4b79637a2745405e812bae07a195cefe
I20260923 22:12:41.086068 5706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:41.088909 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.077045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.1.132.190:35191)
0923 22:12:41.077207 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.077219 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.077246 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:41.077716 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.077723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.077775 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.077897 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.078663 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.079619 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.081185 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.081686 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.081815 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.084904 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.084908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.084909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:41.084921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.086883 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.087368 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.087372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.087372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.087479 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.087658 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.087659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.087659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.088051 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.088243 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.088546 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.088695 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260923 22:12:41.106146 5706 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:41.106937 5706 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:41.121666 5706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: Generated new TSK 0
I20260923 22:12:41.121912 5706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:41.151721 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:41.245424 5718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.245652 5718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.245678 5718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.246915 5718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.246966 5718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.246999 5718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:41.248693 5718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:41.248942 5718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.249164 5718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.254742 5723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.254737 5726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.254854 5724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.254812 5718 server_base.cc:1034] running on GCE node
I20260923 22:12:41.255867 5718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.256166 5718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.257355 5718 hybrid_clock.cc:648] HybridClock initialized: now 1790201561257332 us; error 38 us; skew 500 ppm
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:41.260443 5718 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:41.260842 5718 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.260900 5718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.261026 5718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.262233 5718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "92ab245225b04320a7b5d3ee83e42ebe"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "9a18b3c2cc4b6570e308944ad8024d55"
server_key_iv: "49371fdf2af272cc2ff040da65a31d1b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.262589 5718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "92ab245225b04320a7b5d3ee83e42ebe"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "9a18b3c2cc4b6570e308944ad8024d55"
server_key_iv: "49371fdf2af272cc2ff040da65a31d1b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.263900 5718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:41.264699 5732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.264858 5718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.264937 5718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "92ab245225b04320a7b5d3ee83e42ebe"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "9a18b3c2cc4b6570e308944ad8024d55"
server_key_iv: "49371fdf2af272cc2ff040da65a31d1b"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.265007 5718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.274988 5718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.277164 5718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.277304 5718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.277529 5718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:41.277817 5718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:41.277851 5718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.277881 5718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:41.277904 5718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.283337 5718 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:36721
I20260923 22:12:41.283396 5845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:36721 every 8 connection(s)
I20260923 22:12:41.283644 5718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:41.288949 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5718
I20260923 22:12:41.289060 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:41.289237 1554 external_mini_cluster.cc:1428] Setting key b03299e8e6614f5ac922be60f228677f
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:41.292873 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.284326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:47109 (local address 127.1.132.190:35191)
0923 22:12:41.284384 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.284386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.284608 (+ 222us) server_negotiation.cc:407] Connection header received
0923 22:12:41.285109 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.285110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.285122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.285173 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.286056 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.286561 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.287542 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.287832 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.287889 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.290158 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.290159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.290159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:41.290166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.291382 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.291850 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.291851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.291851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.291910 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.292210 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.292212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.292212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.292365 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.292449 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.292694 (+ 245us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.292786 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:41.292953 5848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.284335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35191 (local address 127.1.132.129:47109)
0923 22:12:41.284493 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.284530 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.285016 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.285245 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.285249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.285502 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:41.285963 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.285968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.286646 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.286648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.287458 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.287460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.287910 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.287910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.288037 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.288523 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:41.288531 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:41.290061 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:41.291454 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.291461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.291463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.291761 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.291979 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.291980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.291980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.292109 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.292436 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:41.292440 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:41.292561 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.292654 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.292782 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260923 22:12:41.293365 5846 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35191
I20260923 22:12:41.293463 5846 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:41.293609 5846 heartbeater.cc:511] Master 127.1.132.190:35191 requested a full tablet report, sending...
I20260923 22:12:41.294072 5656 ts_manager.cc:194] Registered new tserver with Master: 92ab245225b04320a7b5d3ee83e42ebe (127.1.132.129:36721)
I20260923 22:12:41.294777 5656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:47109
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:41.315785 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:41.399300 5851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.399597 5851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.399629 5851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.400940 5851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.400992 5851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.401021 5851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:41.402632 5851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:41.402868 5851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.403084 5851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.408396 5856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.408401 5859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.408401 5857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.408429 5851 server_base.cc:1034] running on GCE node
I20260923 22:12:41.408943 5851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.409262 5851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.410429 5851 hybrid_clock.cc:648] HybridClock initialized: now 1790201561410388 us; error 68 us; skew 500 ppm
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:41.413012 5851 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:41.413316 5851 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.413360 5851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.413451 5851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.414500 5851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "9f205a8bc2054690a6830b208fbdca92"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d125433d870f69951130bd06ad5e7618"
server_key_iv: "e541b0974862fc7081a87b2262413ce2"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.414804 5851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "9f205a8bc2054690a6830b208fbdca92"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d125433d870f69951130bd06ad5e7618"
server_key_iv: "e541b0974862fc7081a87b2262413ce2"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.416143 5851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:41.416955 5865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.417140 5851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:41.417238 5851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "9f205a8bc2054690a6830b208fbdca92"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "d125433d870f69951130bd06ad5e7618"
server_key_iv: "e541b0974862fc7081a87b2262413ce2"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.417316 5851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.446884 5851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.449039 5851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.449173 5851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.449393 5851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:41.449720 5851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:41.449755 5851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.449779 5851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:41.449801 5851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.455260 5851 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:35825
I20260923 22:12:41.455327 5978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:35825 every 8 connection(s)
I20260923 22:12:41.455546 5851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:41.462844 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5851
I20260923 22:12:41.462961 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:41.463148 1554 external_mini_cluster.cc:1428] Setting key fb0f6917ad2543bf3b1a972c87745c32
I20260923 22:12:41.465438 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.456252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:47301 (local address 127.1.132.190:35191)
0923 22:12:41.456301 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.456303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.456523 (+ 220us) server_negotiation.cc:407] Connection header received
0923 22:12:41.457039 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.457040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.457053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.457104 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.458037 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.458541 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.459564 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.459850 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.459939 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.462369 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.462371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.462371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:41.462379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.464014 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.464470 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.464472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.464472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.464536 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.464779 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.464780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.464780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.464898 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.464953 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.465252 (+ 299us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.465346 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:41.465624 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.456235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35191 (local address 127.1.132.130:47301)
0923 22:12:41.456402 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.456440 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.456939 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.457187 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.457191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.457460 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:41.457938 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.457943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.458631 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.458633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.459479 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.459482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.459969 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.459970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.460115 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.460692 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:41.460701 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:41.462252 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:41.464090 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.464100 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.464103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.464402 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.464592 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.464593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.464594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.464715 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.464953 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:41.464961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:41.465113 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.465207 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.465395 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260923 22:12:41.466254 5979 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35191
I20260923 22:12:41.466353 5979 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:41.466504 5979 heartbeater.cc:511] Master 127.1.132.190:35191 requested a full tablet report, sending...
I20260923 22:12:41.466890 5656 ts_manager.cc:194] Registered new tserver with Master: 9f205a8bc2054690a6830b208fbdca92 (127.1.132.130:35825)
I20260923 22:12:41.467381 5656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:47301
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:41.488886 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:41.568245 5984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:41.568470 5984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:41.568511 5984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:41.569756 5984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:41.569808 5984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:41.569844 5984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:41.571465 5984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:41.571732 5984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:41.571943 5984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:41.576725 5989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.576824 5984 server_base.cc:1034] running on GCE node
W20260923 22:12:41.576711 5992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:41.576714 5990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:41.577152 5984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:41.577391 5984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:41.578555 5984 hybrid_clock.cc:648] HybridClock initialized: now 1790201561578542 us; error 27 us; skew 500 ppm
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:41.581097 5984 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:41.581458 5984 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:41.581543 5984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:41.581676 5984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:41.582778 5984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "1f8a6262b6df4e148f7a5b06185daf71"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "7c02275c0c45a4e00940a81d085d1a45"
server_key_iv: "3d5e9059961188b13e80e45491eb2c30"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.583102 5984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "1f8a6262b6df4e148f7a5b06185daf71"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "7c02275c0c45a4e00940a81d085d1a45"
server_key_iv: "3d5e9059961188b13e80e45491eb2c30"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.584364 5984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:41.585036 5998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.585191 5984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:41.585270 5984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "1f8a6262b6df4e148f7a5b06185daf71"
format_stamp: "Formatted at 2026-09-23 22:12:41 on dist-test-slave-l4q5"
server_key: "7c02275c0c45a4e00940a81d085d1a45"
server_key_iv: "3d5e9059961188b13e80e45491eb2c30"
server_key_version: "encryptionkey@0"
I20260923 22:12:41.585330 5984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:41.595620 5984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:41.597695 5984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:41.597833 5984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:41.598062 5984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:41.598357 5984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:41.598390 5984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.598418 5984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:41.598443 5984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:41.603945 5984 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:34203
I20260923 22:12:41.604032 6111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:34203 every 8 connection(s)
I20260923 22:12:41.604343 5984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:12:41 dist-test-slave-l4q5 krb5kdc[5612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:41.613363 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.605071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:45739 (local address 127.1.132.190:35191)
0923 22:12:41.605155 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.605157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.605323 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:12:41.605847 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.605848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.605861 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.605920 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.606842 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.607352 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.608413 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.608614 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.608671 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.610788 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.610790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.610790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:41.610797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.612038 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.612585 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.612586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.612587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.612648 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.612855 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.612856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.612856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.612964 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.613011 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.613231 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.613296 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 22:12:41.613572 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.605026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35191 (local address 127.1.132.131:45739)
0923 22:12:41.605189 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.605237 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.605745 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.606003 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.606007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.606273 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:41.606745 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.606750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.607442 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.607443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.608337 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.608339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.608687 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.608687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.608821 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.609305 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:41.609313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:41.610692 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:41.612249 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.612257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.612260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.612503 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.612705 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:41.612705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:41.612706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:41.612808 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:41.613005 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:41.613009 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:41.613135 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.613216 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.613339 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260923 22:12:41.613683 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 5984
I20260923 22:12:41.613781 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:41.613935 1554 external_mini_cluster.cc:1428] Setting key 56280d76266f8eca236a82372277306f
I20260923 22:12:41.614319 6112 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35191
I20260923 22:12:41.614415 6112 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:41.614630 6112 heartbeater.cc:511] Master 127.1.132.190:35191 requested a full tablet report, sending...
I20260923 22:12:41.615002 5656 ts_manager.cc:194] Registered new tserver with Master: 1f8a6262b6df4e148f7a5b06185daf71 (127.1.132.131:34203)
I20260923 22:12:41.615424 5656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:45739
I20260923 22:12:41.625702 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:41.633240 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.626950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.1.132.190:35191)
0923 22:12:41.626995 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.626997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.627087 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:12:41.627135 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.627136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.627145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.627191 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:41.627713 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.628263 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.629082 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.629284 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.629346 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.630719 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:41.630720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:41.630720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:41.630727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:41.632049 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.632436 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.632437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.632437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.632493 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:41.632733 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:41.632734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:41.632734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:41.632885 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:41.632960 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.633107 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.633173 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:41.634999 5656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:41.636798 5656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:41.642985 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:36721 (local address 127.0.0.1:54174)
0923 22:12:41.639504 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.639512 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.639619 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.639915 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.639918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.639920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:41.640007 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.640009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.640976 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.640977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.641918 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.641921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.642633 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.642634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.642734 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.642750 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.642824 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.642891 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260923 22:12:41.643007 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.1.132.129:36721)
0923 22:12:41.639433 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.639449 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.639659 (+ 210us) server_negotiation.cc:407] Connection header received
0923 22:12:41.639712 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.639714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.639730 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.639821 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:41.640082 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.640880 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.642003 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.642550 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.642618 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.642683 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.642870 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.642922 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260923 22:12:41.643925 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:35825 (local address 127.0.0.1:35930)
0923 22:12:41.639513 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.639522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.639591 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.639956 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.639956 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.639958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:41.640005 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.640007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.641378 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.641379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.642661 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.642664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.643674 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.643676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.643742 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.643746 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.643794 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.643842 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":64,"spinlock_wait_cycles":384,"thread_start_us":96,"threads_started":1}
I20260923 22:12:41.643929 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.1.132.130:35825)
0923 22:12:41.639433 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.639446 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.639671 (+ 225us) server_negotiation.cc:407] Connection header received
0923 22:12:41.639712 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.639714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.639730 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.639805 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:41.640084 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.641265 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.642759 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.643584 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.643656 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.643717 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.643807 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.643848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260923 22:12:41.643864 5780 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d (DEFAULT_TABLE table=test-table [id=40615652d218450a9e8ffabacdebe453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:41.644349 5780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba9d7f0e9fd4ad79f7676b65925025d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.644591 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:34203 (local address 127.0.0.1:51090)
0923 22:12:41.639461 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.639505 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.639620 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.639922 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.639923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.639926 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:41.640005 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.640007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.641387 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.641388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.642662 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.642664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.644338 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.644339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.644418 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.644423 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.644471 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.644522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":71,"thread_start_us":49,"threads_started":1}
I20260923 22:12:41.644752 5913 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d (DEFAULT_TABLE table=test-table [id=40615652d218450a9e8ffabacdebe453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:41.645099 5913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba9d7f0e9fd4ad79f7676b65925025d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.647956 6129 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Bootstrap starting.
I20260923 22:12:41.648458 6130 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Bootstrap starting.
I20260923 22:12:41.648733 6129 tablet_bootstrap.cc:654] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.648769 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.639124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.1.132.131:34203)
0923 22:12:41.639284 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.639289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.639616 (+ 327us) server_negotiation.cc:407] Connection header received
0923 22:12:41.639734 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.639738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.639754 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.639822 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:41.640101 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.641262 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.642777 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.643921 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.648470 (+ 4549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.648580 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.648615 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.648675 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260923 22:12:41.649101 6130 tablet_bootstrap.cc:654] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.649358 6129 log.cc:824] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.649492 6046 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d (DEFAULT_TABLE table=test-table [id=40615652d218450a9e8ffabacdebe453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:41.649634 6130 log.cc:824] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.649855 6046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba9d7f0e9fd4ad79f7676b65925025d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:41.650177 6129 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: No bootstrap required, opened a new log
I20260923 22:12:41.650249 6129 ts_tablet_manager.cc:1397] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:41.650308 6130 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: No bootstrap required, opened a new log
I20260923 22:12:41.650367 6130 ts_tablet_manager.cc:1397] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:41.652029 6130 raft_consensus.cc:348] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.652170 6130 raft_consensus.cc:374] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.652243 6130 raft_consensus.cc:729] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f205a8bc2054690a6830b208fbdca92, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.652354 6130 consensus_queue.cc:260] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.652346 6129 raft_consensus.cc:348] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.652521 6129 raft_consensus.cc:374] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.652570 6129 raft_consensus.cc:729] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ab245225b04320a7b5d3ee83e42ebe, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.652650 6130 ts_tablet_manager.cc:1428] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:12:41.652727 6129 consensus_queue.cc:260] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.652796 5979 heartbeater.cc:503] Master 127.1.132.190:35191 was elected leader, sending a full tablet report...
I20260923 22:12:41.653010 5846 heartbeater.cc:503] Master 127.1.132.190:35191 was elected leader, sending a full tablet report...
I20260923 22:12:41.653012 6129 ts_tablet_manager.cc:1428] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260923 22:12:41.653599 6135 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Bootstrap starting.
I20260923 22:12:41.654232 6135 tablet_bootstrap.cc:654] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:41.654779 6135 log.cc:824] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:41.655575 6135 tablet_bootstrap.cc:492] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: No bootstrap required, opened a new log
I20260923 22:12:41.655637 6135 ts_tablet_manager.cc:1397] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:41.657089 6135 raft_consensus.cc:348] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.657233 6135 raft_consensus.cc:374] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:41.657255 6135 raft_consensus.cc:729] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f8a6262b6df4e148f7a5b06185daf71, State: Initialized, Role: FOLLOWER
I20260923 22:12:41.657364 6135 consensus_queue.cc:260] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.657640 6135 ts_tablet_manager.cc:1428] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:41.657711 6112 heartbeater.cc:503] Master 127.1.132.190:35191 was elected leader, sending a full tablet report...
W20260923 22:12:41.706104 5980 tablet.cc:2369] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:12:41.784351 5847 tablet.cc:2369] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:41.819321 6137 raft_consensus.cc:482] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:41.819484 6137 raft_consensus.cc:504] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.819869 6137 leader_election.cc:288] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f205a8bc2054690a6830b208fbdca92 (127.1.132.130:35825), 92ab245225b04320a7b5d3ee83e42ebe (127.1.132.129:36721)
I20260923 22:12:41.823745 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.820139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:34215 (local address 127.1.132.129:36721)
0923 22:12:41.820206 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.820208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.820345 (+ 137us) server_negotiation.cc:407] Connection header received
0923 22:12:41.820373 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.820374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.820382 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.820422 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:41.820558 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.821256 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.822790 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.823348 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.823431 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.823484 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.823629 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.823669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:41.823746 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.820111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:42529 (local address 127.1.132.130:35825)
0923 22:12:41.820156 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.820158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.820173 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:12:41.820228 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.820229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.820238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.820292 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:41.820462 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.821261 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.822795 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.823355 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.823432 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.823442 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.823646 (+ 204us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.823688 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:41.823761 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.820043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:35825 (local address 127.1.132.131:42529)
0923 22:12:41.820098 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.820109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.820152 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.820320 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.820321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.820323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:41.820397 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.820399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.821366 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.821367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.822677 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.822681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.823460 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.823461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.823580 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.823582 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.823632 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.823665 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":55}
I20260923 22:12:41.823776 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.820070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:36721 (local address 127.1.132.131:34215)
0923 22:12:41.820268 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.820277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:41.820332 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:41.820439 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:41.820440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:41.820454 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:41.820505 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.820506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.821366 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.821368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.822677 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:41.822681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.823460 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.823461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:41.823544 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.823549 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:41.823621 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:41.823665 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":5,"thread_start_us":98,"threads_started":1}
I20260923 22:12:41.824156 5800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba9d7f0e9fd4ad79f7676b65925025d" candidate_uuid: "1f8a6262b6df4e148f7a5b06185daf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ab245225b04320a7b5d3ee83e42ebe" is_pre_election: true
I20260923 22:12:41.824288 5933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba9d7f0e9fd4ad79f7676b65925025d" candidate_uuid: "1f8a6262b6df4e148f7a5b06185daf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f205a8bc2054690a6830b208fbdca92" is_pre_election: true
I20260923 22:12:41.824349 5800 raft_consensus.cc:2454] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f8a6262b6df4e148f7a5b06185daf71 in term 0.
I20260923 22:12:41.824478 5933 raft_consensus.cc:2454] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f8a6262b6df4e148f7a5b06185daf71 in term 0.
I20260923 22:12:41.824604 6000 leader_election.cc:302] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f8a6262b6df4e148f7a5b06185daf71, 92ab245225b04320a7b5d3ee83e42ebe; no voters:
I20260923 22:12:41.824753 6137 raft_consensus.cc:2781] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:41.824808 6137 raft_consensus.cc:482] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:41.824824 6137 raft_consensus.cc:3037] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.825507 6137 raft_consensus.cc:504] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.825654 6137 leader_election.cc:288] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [CANDIDATE]: Term 1 election: Requested vote from peers 9f205a8bc2054690a6830b208fbdca92 (127.1.132.130:35825), 92ab245225b04320a7b5d3ee83e42ebe (127.1.132.129:36721)
I20260923 22:12:41.825824 5800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba9d7f0e9fd4ad79f7676b65925025d" candidate_uuid: "1f8a6262b6df4e148f7a5b06185daf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ab245225b04320a7b5d3ee83e42ebe"
I20260923 22:12:41.825822 5933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba9d7f0e9fd4ad79f7676b65925025d" candidate_uuid: "1f8a6262b6df4e148f7a5b06185daf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f205a8bc2054690a6830b208fbdca92"
I20260923 22:12:41.825892 5800 raft_consensus.cc:3037] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.825894 5933 raft_consensus.cc:3037] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:41.826571 5800 raft_consensus.cc:2454] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f8a6262b6df4e148f7a5b06185daf71 in term 1.
I20260923 22:12:41.826653 5933 raft_consensus.cc:2454] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f8a6262b6df4e148f7a5b06185daf71 in term 1.
I20260923 22:12:41.826740 6000 leader_election.cc:302] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f8a6262b6df4e148f7a5b06185daf71, 92ab245225b04320a7b5d3ee83e42ebe; no voters:
I20260923 22:12:41.826825 6137 raft_consensus.cc:2781] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:41.826990 6137 raft_consensus.cc:686] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 1 LEADER]: Becoming Leader. State: Replica: 1f8a6262b6df4e148f7a5b06185daf71, State: Running, Role: LEADER
I20260923 22:12:41.827082 6137 consensus_queue.cc:237] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } }
I20260923 22:12:41.827935 5655 catalog_manager.cc:5495] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f8a6262b6df4e148f7a5b06185daf71 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "1f8a6262b6df4e148f7a5b06185daf71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a6262b6df4e148f7a5b06185daf71" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 34203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 } health_report { overall_health: UNKNOWN } } }
W20260923 22:12:41.855147 6113 tablet.cc:2369] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:41.874401 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.872121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.1.132.131:34203)
0923 22:12:41.872198 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:41.872200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:41.872209 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:41.872251 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:41.872252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:41.872270 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:41.872335 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:41.872491 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.872980 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.873742 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:41.873924 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:41.873972 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:41.874112 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:41.874226 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:41.874285 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:41.874302 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 22:12:41.874319 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:41.876127 5933 raft_consensus.cc:1261] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 1 FOLLOWER]: Refusing update from remote peer 1f8a6262b6df4e148f7a5b06185daf71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:41.876127 5800 raft_consensus.cc:1261] T 7ba9d7f0e9fd4ad79f7676b65925025d P 92ab245225b04320a7b5d3ee83e42ebe [term 1 FOLLOWER]: Refusing update from remote peer 1f8a6262b6df4e148f7a5b06185daf71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:41.876480 6137 consensus_queue.cc:1035] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f205a8bc2054690a6830b208fbdca92" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 35825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:41.876621 6139 consensus_queue.cc:1035] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ab245225b04320a7b5d3ee83e42ebe" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:41.878911 6142 mvcc.cc:205] Tried to move back new op lower bound from 7332665597442084864 to 7332665597244575744. Current Snapshot: MvccSnapshot[applied={T|T < 7332665597442084864}]
I20260923 22:12:41.879630 6143 mvcc.cc:205] Tried to move back new op lower bound from 7332665597442084864 to 7332665597244575744. Current Snapshot: MvccSnapshot[applied={T|T < 7332665597442084864 or (T in {7332665597442084864})}]
I20260923 22:12:41.880380 6145 mvcc.cc:205] Tried to move back new op lower bound from 7332665597442084864 to 7332665597244575744. Current Snapshot: MvccSnapshot[applied={T|T < 7332665597442084864}]
I20260923 22:12:41.881284 5656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:41.881371 5656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:41.882115 5656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 711ead9275c14858acbda2c1922150be: Sending DeleteTablet for 3 replicas of tablet 7ba9d7f0e9fd4ad79f7676b65925025d
I20260923 22:12:41.882437 5913 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260923 22:12:41.882444 6046 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51090
I20260923 22:12:41.882687 6151 tablet_replica.cc:330] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: stopping tablet replica
I20260923 22:12:41.882718 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5718
I20260923 22:12:41.882695 5780 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba9d7f0e9fd4ad79f7676b65925025d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:41 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:54174
I20260923 22:12:41.882786 6151 raft_consensus.cc:2229] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:41.882886 6151 raft_consensus.cc:2258] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:41.883020 6152 tablet_replica.cc:330] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: stopping tablet replica
I20260923 22:12:41.883157 6152 raft_consensus.cc:2229] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:41.883255 6152 raft_consensus.cc:2258] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:41.883615 6152 ts_tablet_manager.cc:1905] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:41.885197 6152 ts_tablet_manager.cc:1918] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:41.885272 6152 log.cc:1196] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/7ba9d7f0e9fd4ad79f7676b65925025d
I20260923 22:12:41.885411 6151 ts_tablet_manager.cc:1905] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:41.885529 6152 ts_tablet_manager.cc:1939] T 7ba9d7f0e9fd4ad79f7676b65925025d P 1f8a6262b6df4e148f7a5b06185daf71: Deleting consensus metadata
I20260923 22:12:41.885919 5642 catalog_manager.cc:4841] TS 1f8a6262b6df4e148f7a5b06185daf71 (127.1.132.131:34203): tablet 7ba9d7f0e9fd4ad79f7676b65925025d (table test-table [id=40615652d218450a9e8ffabacdebe453]) successfully deleted
I20260923 22:12:41.886946 6151 ts_tablet_manager.cc:1918] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:41.887012 6151 log.cc:1196] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/7ba9d7f0e9fd4ad79f7676b65925025d
I20260923 22:12:41.887235 6151 ts_tablet_manager.cc:1939] T 7ba9d7f0e9fd4ad79f7676b65925025d P 9f205a8bc2054690a6830b208fbdca92: Deleting consensus metadata
W20260923 22:12:41.887497 5641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:36721 (error 108)
I20260923 22:12:41.887735 5640 catalog_manager.cc:4841] TS 9f205a8bc2054690a6830b208fbdca92 (127.1.132.130:35825): tablet 7ba9d7f0e9fd4ad79f7676b65925025d (table test-table [id=40615652d218450a9e8ffabacdebe453]) successfully deleted
I20260923 22:12:41.887892 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5851
I20260923 22:12:41.887977 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:41.887863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:36721 (local address 127.0.0.1:54190)
0923 22:12:41.887915 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:41.887946 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:36721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:41.888098 5641 catalog_manager.cc:4584] TS 92ab245225b04320a7b5d3ee83e42ebe (127.1.132.129:36721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ba9d7f0e9fd4ad79f7676b65925025d: Network error: Client connection negotiation failed: client connection to 127.1.132.129:36721: connect: Connection refused (error 111)
I20260923 22:12:41.892769 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5984
I20260923 22:12:41.896945 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 5625
2026-09-23T22:12:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2017 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:41Z Disabled control of system clock
I20260923 22:12:41.923022 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34229
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:34229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:42.001782 6157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.001992 6157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.002010 6157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:42.003093 6157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:42.003126 6157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:42.003139 6157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:42.003149 6157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:42.003159 6157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:42.004464 6157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:34229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:42.004671 6157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:42.004910 6157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:42.009755 6163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.009804 6157 server_base.cc:1034] running on GCE node
W20260923 22:12:42.009784 6165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.009768 6162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.010252 6157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:42.010566 6157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:42.011732 6157 hybrid_clock.cc:648] HybridClock initialized: now 1790201562011714 us; error 34 us; skew 500 ppm
I20260923 22:12:42.013175 6157 webserver.cc:460] Webserver started at http://127.1.132.190:37151/ using document root <none> and password file <none>
I20260923 22:12:42.013410 6157 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:42.013461 6157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:42.013597 6157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:42.014654 6157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "c86d89c4fced474580c33fe59f44e2a6"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "09c0e8f23b8fd43438cc33aedbd08fba"
server_key_iv: "b1e7a57e4f295ed612304512b6a88ccc"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.014968 6157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "c86d89c4fced474580c33fe59f44e2a6"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "09c0e8f23b8fd43438cc33aedbd08fba"
server_key_iv: "b1e7a57e4f295ed612304512b6a88ccc"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.016240 6157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:42.017050 6171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.017227 6157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:42.017320 6157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "c86d89c4fced474580c33fe59f44e2a6"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "09c0e8f23b8fd43438cc33aedbd08fba"
server_key_iv: "b1e7a57e4f295ed612304512b6a88ccc"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.017397 6157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:42.020865 6157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:42.021136 6157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:42.021274 6157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:42.025041 6157 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:34229
I20260923 22:12:42.025131 6223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:34229 every 8 connection(s)
I20260923 22:12:42.025424 6157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:42.026073 6224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:42.027041 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6157
I20260923 22:12:42.027127 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:42.027279 1554 external_mini_cluster.cc:1428] Setting key 23eac2d811a5fe1e12e61984f1faa590
I20260923 22:12:42.029397 6224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: Bootstrap starting.
I20260923 22:12:42.029906 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.027757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.1.132.190:34229)
0923 22:12:42.027997 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.028007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.028040 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:12:42.028620 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.028630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.028687 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.028847 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.029469 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.029473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.029474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.029478 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.029523 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.029537 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.029606 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.029656 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.029715 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
I20260923 22:12:42.030109 6224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:42.030352 6224 log.cc:824] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:42.031131 6224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: No bootstrap required, opened a new log
I20260923 22:12:42.032584 6224 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } }
I20260923 22:12:42.032732 6224 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:42.032764 6224 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86d89c4fced474580c33fe59f44e2a6, State: Initialized, Role: FOLLOWER
I20260923 22:12:42.032881 6224 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } }
I20260923 22:12:42.032958 6224 raft_consensus.cc:388] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:42.032987 6224 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:42.033034 6224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:42.033562 6224 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } }
I20260923 22:12:42.033658 6224 leader_election.cc:302] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c86d89c4fced474580c33fe59f44e2a6; no voters:
I20260923 22:12:42.033821 6224 leader_election.cc:288] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:42.034104 6224 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:42.034101 6229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:42.034253 6229 raft_consensus.cc:686] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [term 1 LEADER]: Becoming Leader. State: Replica: c86d89c4fced474580c33fe59f44e2a6, State: Running, Role: LEADER
I20260923 22:12:42.034354 6229 consensus_queue.cc:237] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } }
I20260923 22:12:42.034706 6230 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c86d89c4fced474580c33fe59f44e2a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } } }
I20260923 22:12:42.034797 6230 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:42.034957 6231 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c86d89c4fced474580c33fe59f44e2a6. Latest consensus state: current_term: 1 leader_uuid: "c86d89c4fced474580c33fe59f44e2a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86d89c4fced474580c33fe59f44e2a6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34229 } } }
I20260923 22:12:42.035023 6231 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:42.034991 6238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:42.035575 6238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:42.037395 6238 catalog_manager.cc:1347] Generated new cluster ID: ca21d4f723154fbf8122de0299e78bab
I20260923 22:12:42.037444 6238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:42.046048 6238 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:42.046242 6238 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:42.049348 6238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: Generated new TSK 0
I20260923 22:12:42.049528 6238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:42.057371 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--builtin_ntp_servers=127.1.132.148:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:42.139119 6248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.139328 6248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.139350 6248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:42.140578 6248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:42.140614 6248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:42.140638 6248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:42.142011 6248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:42.142225 6248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:42.142518 6248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:42.147331 6254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.147337 6253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.147383 6256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.147975 6248 server_base.cc:1034] running on GCE node
I20260923 22:12:42.148242 6248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:42.148500 6248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:42.149642 6248 hybrid_clock.cc:648] HybridClock initialized: now 1790201562149629 us; error 30 us; skew 500 ppm
I20260923 22:12:42.150913 6248 webserver.cc:460] Webserver started at http://127.1.132.129:35473/ using document root <none> and password file <none>
I20260923 22:12:42.151147 6248 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:42.151197 6248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:42.151322 6248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:42.152449 6248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "2c88d500ad6d4b70aca15bf9c180bb9a"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "6da5bdbc84b18ab7b12e657d83115af7"
server_key_iv: "09e9e73f1dde35ab7780cd2cd94ba1be"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.152771 6248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "2c88d500ad6d4b70aca15bf9c180bb9a"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "6da5bdbc84b18ab7b12e657d83115af7"
server_key_iv: "09e9e73f1dde35ab7780cd2cd94ba1be"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.153968 6248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:42.154690 6262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.154857 6248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:42.154937 6248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "2c88d500ad6d4b70aca15bf9c180bb9a"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "6da5bdbc84b18ab7b12e657d83115af7"
server_key_iv: "09e9e73f1dde35ab7780cd2cd94ba1be"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.155005 6248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:42.158322 6248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:42.158568 6248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:42.158687 6248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:42.158867 6248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:42.159176 6248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:42.159209 6248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.159236 6248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:42.159260 6248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.165525 6248 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:35597
I20260923 22:12:42.165575 6375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:35597 every 8 connection(s)
I20260923 22:12:42.165906 6248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:42.168774 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.166628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:37739 (local address 127.1.132.190:34229)
0923 22:12:42.166680 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.166682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.166907 (+ 225us) server_negotiation.cc:407] Connection header received
0923 22:12:42.167351 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.167353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.167358 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.167407 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.168584 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.168585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.168585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.168589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.168603 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.168606 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.168649 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.168709 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.168721 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:12:42.168946 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.166603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34229 (local address 127.1.132.129:37739)
0923 22:12:42.166789 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.166825 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.167268 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.167471 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.167474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.167482 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.168444 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.168444 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.168474 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.168505 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.168650 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.168652 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.168654 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.168705 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.168758 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260923 22:12:42.169447 6376 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34229
I20260923 22:12:42.169538 6376 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:42.169685 6376 heartbeater.cc:511] Master 127.1.132.190:34229 requested a full tablet report, sending...
I20260923 22:12:42.170140 6188 ts_manager.cc:194] Registered new tserver with Master: 2c88d500ad6d4b70aca15bf9c180bb9a (127.1.132.129:35597)
I20260923 22:12:42.171954 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6248
I20260923 22:12:42.172031 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:42.172214 1554 external_mini_cluster.cc:1428] Setting key 478f9796ae9ba09d9b044f57a93b70dd
I20260923 22:12:42.173017 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--builtin_ntp_servers=127.1.132.148:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:42.252097 6379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.252334 6379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.252357 6379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:42.253557 6379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:42.253604 6379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:42.253631 6379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:42.254982 6379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:42.255213 6379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:42.255498 6379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:42.260306 6387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.260310 6384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.260314 6385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.260574 6379 server_base.cc:1034] running on GCE node
I20260923 22:12:42.260797 6379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:42.261014 6379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:42.262140 6379 hybrid_clock.cc:648] HybridClock initialized: now 1790201562262127 us; error 31 us; skew 500 ppm
I20260923 22:12:42.263557 6379 webserver.cc:460] Webserver started at http://127.1.132.130:39807/ using document root <none> and password file <none>
I20260923 22:12:42.263805 6379 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:42.263854 6379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:42.263949 6379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:42.265015 6379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "8c3ba266eaa646ac8fc7f0d3090dd275"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "d9843e7e58baa7decb2e8bf449bb7af5"
server_key_iv: "9813205f2d897b83d81c1ed3da658317"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.265319 6379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "8c3ba266eaa646ac8fc7f0d3090dd275"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "d9843e7e58baa7decb2e8bf449bb7af5"
server_key_iv: "9813205f2d897b83d81c1ed3da658317"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.266606 6379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:42.267330 6393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.267511 6379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:42.267599 6379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "8c3ba266eaa646ac8fc7f0d3090dd275"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "d9843e7e58baa7decb2e8bf449bb7af5"
server_key_iv: "9813205f2d897b83d81c1ed3da658317"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.267676 6379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:42.270917 6379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:42.271171 6379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:42.271286 6379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:42.271482 6379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:42.271811 6379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:42.271854 6379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.271888 6379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:42.271908 6379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.277333 6379 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:46445
I20260923 22:12:42.277380 6506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:46445 every 8 connection(s)
I20260923 22:12:42.277709 6379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:42.280956 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.278482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:47107 (local address 127.1.132.190:34229)
0923 22:12:42.278572 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.278574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.278876 (+ 302us) server_negotiation.cc:407] Connection header received
0923 22:12:42.279353 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.279355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.279361 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.279413 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.280718 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.280720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.280720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.280725 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.280741 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.280745 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.280796 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.280865 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.280879 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 22:12:42.281164 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.278415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34229 (local address 127.1.132.130:47107)
0923 22:12:42.278720 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.278762 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.279258 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.279498 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.279503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.279513 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.280556 (+ 1043us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.280557 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.280589 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.280622 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.280795 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.280800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.280804 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.280873 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.280938 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260923 22:12:42.281721 6507 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34229
I20260923 22:12:42.281803 6507 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:42.281934 6507 heartbeater.cc:511] Master 127.1.132.190:34229 requested a full tablet report, sending...
I20260923 22:12:42.282316 6188 ts_manager.cc:194] Registered new tserver with Master: 8c3ba266eaa646ac8fc7f0d3090dd275 (127.1.132.130:46445)
I20260923 22:12:42.287420 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6379
I20260923 22:12:42.287566 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:42.287738 1554 external_mini_cluster.cc:1428] Setting key f3ae145472908df4e104a1de639150df
I20260923 22:12:42.288872 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--builtin_ntp_servers=127.1.132.148:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:12:42.371273 6510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.371501 6510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.371523 6510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:42.372735 6510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:42.372776 6510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:42.372802 6510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:42.374197 6510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:42.374398 6510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:42.374656 6510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:42.379527 6515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.379657 6518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.379557 6516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.379631 6510 server_base.cc:1034] running on GCE node
I20260923 22:12:42.379989 6510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:42.380285 6510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:42.381462 6510 hybrid_clock.cc:648] HybridClock initialized: now 1790201562381435 us; error 35 us; skew 500 ppm
I20260923 22:12:42.382881 6510 webserver.cc:460] Webserver started at http://127.1.132.131:35665/ using document root <none> and password file <none>
I20260923 22:12:42.383121 6510 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:42.383170 6510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:42.383280 6510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:42.384399 6510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "b9c412a9d0f84ce98c9a788a73aa1e13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "40efeabf1134ccf580abae312e6ccef2"
server_key_iv: "a5be44c1b2a832f2eb82a274f65737fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.384735 6510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "b9c412a9d0f84ce98c9a788a73aa1e13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "40efeabf1134ccf580abae312e6ccef2"
server_key_iv: "a5be44c1b2a832f2eb82a274f65737fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.385972 6510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:42.386705 6524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.386842 6510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:42.386924 6510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "b9c412a9d0f84ce98c9a788a73aa1e13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "40efeabf1134ccf580abae312e6ccef2"
server_key_iv: "a5be44c1b2a832f2eb82a274f65737fd"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.386982 6510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:42.390430 6510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:42.390676 6510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:42.390787 6510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:42.390969 6510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:42.391245 6510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:42.391276 6510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.391309 6510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:42.391332 6510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.396639 6510 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:44557
I20260923 22:12:42.396710 6637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:44557 every 8 connection(s)
I20260923 22:12:42.397007 6510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:42.400144 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.397762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59965 (local address 127.1.132.190:34229)
0923 22:12:42.397839 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.397841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.398033 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:12:42.398618 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.398619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.398626 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.398708 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.399902 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.399903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.399903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.399908 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.399925 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.399929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.399980 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.400054 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.400070 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:12:42.400363 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.397724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34229 (local address 127.1.132.131:59965)
0923 22:12:42.397895 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.397935 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.398523 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.398783 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.398787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.398797 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.399747 (+ 950us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.399748 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.399778 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.399807 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.399987 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.399992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.399995 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.400066 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.400130 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260923 22:12:42.401113 6638 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34229
I20260923 22:12:42.401213 6638 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:42.401337 6638 heartbeater.cc:511] Master 127.1.132.190:34229 requested a full tablet report, sending...
I20260923 22:12:42.401701 6188 ts_manager.cc:194] Registered new tserver with Master: b9c412a9d0f84ce98c9a788a73aa1e13 (127.1.132.131:44557)
I20260923 22:12:42.403595 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6510
I20260923 22:12:42.403687 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:42.403838 1554 external_mini_cluster.cc:1428] Setting key 6ac5c0953b1ee6dfaa81841b0446e4d8
I20260923 22:12:42.404685 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:12:42.407388 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.406206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.1.132.190:34229)
0923 22:12:42.406260 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.406262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.406490 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:12:42.406566 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.406568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.406574 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.406623 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.407178 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.407179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.407179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.407184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.407199 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.407203 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.407251 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.407298 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.407312 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:42.409330 6188 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:42.411545 6188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:42.416056 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:35597 (local address 127.0.0.1:50916)
0923 22:12:42.414696 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.414703 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.414744 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.414967 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.414973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.414981 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.415489 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.415489 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.415507 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.415533 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.415900 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.415907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.415910 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.415966 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.415979 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260923 22:12:42.416072 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.1.132.129:35597)
0923 22:12:42.414526 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.414530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.414809 (+ 279us) server_negotiation.cc:407] Connection header received
0923 22:12:42.414842 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.414845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.414855 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.415213 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.415685 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.415689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.415690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.415693 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.415723 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.415739 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.415795 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.415981 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.415996 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":78,"threads_started":1}
I20260923 22:12:42.416177 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:44557 (local address 127.0.0.1:57618)
0923 22:12:42.414440 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.414485 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.414546 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.415069 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.415070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.415074 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.415819 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.415820 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.415825 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.415833 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.416054 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.416055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.416055 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.416084 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.416101 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260923 22:12:42.416285 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.1.132.131:44557)
0923 22:12:42.414914 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.414918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.414927 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:42.414961 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.414963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.414972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.415251 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.415907 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.415915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.415915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.415925 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.415953 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.415980 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.416140 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.416161 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.416174 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":71,"threads_started":1}
I20260923 22:12:42.416540 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.1.132.130:46445)
0923 22:12:42.414432 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.414438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.414658 (+ 220us) server_negotiation.cc:407] Connection header received
0923 22:12:42.414689 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.414691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.414703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.414774 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.415871 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.415882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.415883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.415887 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.415919 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.415950 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.416013 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.416445 (+ 432us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.416462 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260923 22:12:42.416904 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.414051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:46445 (local address 127.0.0.1:54380)
0923 22:12:42.414232 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.414600 (+ 368us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.414633 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.415080 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.415080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.415082 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.415766 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.415767 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.415775 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.415788 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.416040 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.416041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.416041 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.416085 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.416099 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260923 22:12:42.416960 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b (DEFAULT_TABLE table=test-table [id=7b3092a92f3447889cf354f00212abce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:42.417119 6310 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b (DEFAULT_TABLE table=test-table [id=7b3092a92f3447889cf354f00212abce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:42.417335 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba7c3c263fa45b2b997ea1010f2968b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:42.417454 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba7c3c263fa45b2b997ea1010f2968b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:42.417536 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b (DEFAULT_TABLE table=test-table [id=7b3092a92f3447889cf354f00212abce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:42.417840 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ba7c3c263fa45b2b997ea1010f2968b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:42.420122 6652 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: Bootstrap starting.
I20260923 22:12:42.420511 6653 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Bootstrap starting.
I20260923 22:12:42.420965 6654 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Bootstrap starting.
I20260923 22:12:42.421018 6652 tablet_bootstrap.cc:654] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:42.421115 6653 tablet_bootstrap.cc:654] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:42.421345 6652 log.cc:824] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:42.421423 6653 log.cc:824] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:42.421541 6654 tablet_bootstrap.cc:654] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:42.421841 6654 log.cc:824] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:42.422075 6652 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: No bootstrap required, opened a new log
I20260923 22:12:42.422115 6653 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: No bootstrap required, opened a new log
I20260923 22:12:42.422142 6652 ts_tablet_manager.cc:1397] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:42.422178 6653 ts_tablet_manager.cc:1397] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:42.422631 6654 tablet_bootstrap.cc:492] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: No bootstrap required, opened a new log
I20260923 22:12:42.422729 6654 ts_tablet_manager.cc:1397] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:12:42.424212 6653 raft_consensus.cc:348] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.424369 6653 raft_consensus.cc:374] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:42.424413 6653 raft_consensus.cc:729] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c412a9d0f84ce98c9a788a73aa1e13, State: Initialized, Role: FOLLOWER
I20260923 22:12:42.424557 6653 consensus_queue.cc:260] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.424630 6652 raft_consensus.cc:348] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.424798 6653 ts_tablet_manager.cc:1428] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:42.424813 6652 raft_consensus.cc:374] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:42.424857 6652 raft_consensus.cc:729] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c88d500ad6d4b70aca15bf9c180bb9a, State: Initialized, Role: FOLLOWER
I20260923 22:12:42.424896 6638 heartbeater.cc:503] Master 127.1.132.190:34229 was elected leader, sending a full tablet report...
I20260923 22:12:42.424969 6652 consensus_queue.cc:260] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.425225 6652 ts_tablet_manager.cc:1428] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:42.425339 6376 heartbeater.cc:503] Master 127.1.132.190:34229 was elected leader, sending a full tablet report...
I20260923 22:12:42.425516 6654 raft_consensus.cc:348] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.425665 6654 raft_consensus.cc:374] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:42.425702 6654 raft_consensus.cc:729] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3ba266eaa646ac8fc7f0d3090dd275, State: Initialized, Role: FOLLOWER
I20260923 22:12:42.425817 6654 consensus_queue.cc:260] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.426046 6654 ts_tablet_manager.cc:1428] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:42.426164 6507 heartbeater.cc:503] Master 127.1.132.190:34229 was elected leader, sending a full tablet report...
W20260923 22:12:42.528306 6508 tablet.cc:2369] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:42.596673 6660 raft_consensus.cc:482] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:42.596822 6660 raft_consensus.cc:504] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.597177 6660 leader_election.cc:288] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9c412a9d0f84ce98c9a788a73aa1e13 (127.1.132.131:44557), 2c88d500ad6d4b70aca15bf9c180bb9a (127.1.132.129:35597)
I20260923 22:12:42.598425 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.597369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:44557 (local address 127.1.132.130:39739)
0923 22:12:42.597420 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.597433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.597477 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.597648 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.597650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.597653 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.598058 (+ 405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.598059 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.598068 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.598081 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.598295 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.598296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.598297 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.598345 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.598358 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:42.598433 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.597459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:39739 (local address 127.1.132.131:44557)
0923 22:12:42.597501 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.597502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.597513 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:42.597540 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.597541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.597546 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.597607 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.598190 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.598191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.598192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.598197 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.598219 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.598225 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.598298 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.598348 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.598362 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:42.598544 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.597491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:56419 (local address 127.1.132.129:35597)
0923 22:12:42.597532 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.597533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.597631 (+ 98us) server_negotiation.cc:407] Connection header received
0923 22:12:42.597664 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.597665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.597670 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.597722 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.598338 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.598340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.598340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.598345 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.598363 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.598368 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.598422 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.598470 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.598483 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:42.598569 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.597372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:35597 (local address 127.1.132.130:56419)
0923 22:12:42.597562 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.597570 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.597611 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.597733 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.597734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.597736 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:42.598238 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:12:42.598238 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:42.598245 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:12:42.598256 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:42.598425 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:42.598425 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:42.598426 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:42.598470 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:12:42.598482 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":40,"thread_start_us":94,"threads_started":1}
I20260923 22:12:42.598800 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba7c3c263fa45b2b997ea1010f2968b" candidate_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" is_pre_election: true
I20260923 22:12:42.598899 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba7c3c263fa45b2b997ea1010f2968b" candidate_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" is_pre_election: true
I20260923 22:12:42.598951 6592 raft_consensus.cc:2454] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3ba266eaa646ac8fc7f0d3090dd275 in term 0.
I20260923 22:12:42.599040 6330 raft_consensus.cc:2454] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3ba266eaa646ac8fc7f0d3090dd275 in term 0.
I20260923 22:12:42.599277 6397 leader_election.cc:302] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c88d500ad6d4b70aca15bf9c180bb9a, 8c3ba266eaa646ac8fc7f0d3090dd275; no voters:
I20260923 22:12:42.599429 6660 raft_consensus.cc:2781] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:42.599495 6660 raft_consensus.cc:482] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:42.599516 6660 raft_consensus.cc:3037] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:42.600246 6660 raft_consensus.cc:504] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.600384 6660 leader_election.cc:288] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [CANDIDATE]: Term 1 election: Requested vote from peers b9c412a9d0f84ce98c9a788a73aa1e13 (127.1.132.131:44557), 2c88d500ad6d4b70aca15bf9c180bb9a (127.1.132.129:35597)
I20260923 22:12:42.600513 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba7c3c263fa45b2b997ea1010f2968b" candidate_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13"
I20260923 22:12:42.600584 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ba7c3c263fa45b2b997ea1010f2968b" candidate_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a"
I20260923 22:12:42.600595 6592 raft_consensus.cc:3037] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:42.600670 6330 raft_consensus.cc:3037] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:42.601285 6592 raft_consensus.cc:2454] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3ba266eaa646ac8fc7f0d3090dd275 in term 1.
I20260923 22:12:42.601367 6330 raft_consensus.cc:2454] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3ba266eaa646ac8fc7f0d3090dd275 in term 1.
I20260923 22:12:42.601483 6394 leader_election.cc:302] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3ba266eaa646ac8fc7f0d3090dd275, b9c412a9d0f84ce98c9a788a73aa1e13; no voters:
I20260923 22:12:42.601567 6660 raft_consensus.cc:2781] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:42.601708 6660 raft_consensus.cc:686] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3ba266eaa646ac8fc7f0d3090dd275, State: Running, Role: LEADER
I20260923 22:12:42.601797 6660 consensus_queue.cc:237] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } }
I20260923 22:12:42.602577 6188 catalog_manager.cc:5495] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3ba266eaa646ac8fc7f0d3090dd275 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3ba266eaa646ac8fc7f0d3090dd275" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:42.647323 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.646308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.1.132.130:46445)
0923 22:12:42.646374 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.646376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.646383 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:42.646421 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.646422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.646436 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.646484 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.647156 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.647157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.647157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.647159 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.647171 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.647174 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.647210 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.647240 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.647251 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260923 22:12:42.647544 6639 tablet.cc:2369] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:42.649317 6592 raft_consensus.cc:1261] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 1 FOLLOWER]: Refusing update from remote peer 8c3ba266eaa646ac8fc7f0d3090dd275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:42.649317 6330 raft_consensus.cc:1261] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 1 FOLLOWER]: Refusing update from remote peer 8c3ba266eaa646ac8fc7f0d3090dd275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:42.649708 6662 consensus_queue.cc:1035] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c412a9d0f84ce98c9a788a73aa1e13" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:42.649838 6660 consensus_queue.cc:1035] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c88d500ad6d4b70aca15bf9c180bb9a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 35597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:42.655094 6187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:42.655208 6187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:42.656167 6187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c86d89c4fced474580c33fe59f44e2a6: Sending DeleteTablet for 3 replicas of tablet 3ba7c3c263fa45b2b997ea1010f2968b
I20260923 22:12:42.656571 6310 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:42 UTC) from {username='slave'} at 127.0.0.1:50916
I20260923 22:12:42.656687 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:42 UTC) from {username='slave'} at 127.0.0.1:57618
I20260923 22:12:42.656785 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ba7c3c263fa45b2b997ea1010f2968b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:42 UTC) from {username='slave'} at 127.0.0.1:54380
I20260923 22:12:42.656956 6675 tablet_replica.cc:330] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a: stopping tablet replica
I20260923 22:12:42.657029 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6248
I20260923 22:12:42.657039 6676 tablet_replica.cc:330] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: stopping tablet replica
I20260923 22:12:42.657095 6675 raft_consensus.cc:2229] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:42.657117 6676 raft_consensus.cc:2229] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:42.657208 6675 raft_consensus.cc:2258] T 3ba7c3c263fa45b2b997ea1010f2968b P 2c88d500ad6d4b70aca15bf9c180bb9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:42.657231 6676 raft_consensus.cc:2258] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:42.659668 6676 ts_tablet_manager.cc:1905] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:42.661825 6676 ts_tablet_manager.cc:1918] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:42.661916 6676 log.cc:1196] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/3ba7c3c263fa45b2b997ea1010f2968b
I20260923 22:12:42.662191 6676 ts_tablet_manager.cc:1939] T 3ba7c3c263fa45b2b997ea1010f2968b P 8c3ba266eaa646ac8fc7f0d3090dd275: Deleting consensus metadata
I20260923 22:12:42.663125 6173 catalog_manager.cc:4841] TS 8c3ba266eaa646ac8fc7f0d3090dd275 (127.1.132.130:46445): tablet 3ba7c3c263fa45b2b997ea1010f2968b (table test-table [id=7b3092a92f3447889cf354f00212abce]) successfully deleted
W20260923 22:12:42.663462 6175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:35597 (error 108)
I20260923 22:12:42.664062 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.663875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:35597 (local address 127.0.0.1:50930)
0923 22:12:42.663942 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.664027 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:12:42.664175 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6379
I20260923 22:12:42.664840 6677 tablet_replica.cc:330] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: stopping tablet replica
I20260923 22:12:42.664954 6677 raft_consensus.cc:2229] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:42.665055 6677 raft_consensus.cc:2258] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:12:42.665421 6175 catalog_manager.cc:4584] TS 2c88d500ad6d4b70aca15bf9c180bb9a (127.1.132.129:35597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ba7c3c263fa45b2b997ea1010f2968b: Network error: Client connection negotiation failed: client connection to 127.1.132.129:35597: connect: Connection refused (error 111)
I20260923 22:12:42.666014 6677 ts_tablet_manager.cc:1905] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:42.667914 6677 ts_tablet_manager.cc:1918] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:42.668002 6677 log.cc:1196] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/3ba7c3c263fa45b2b997ea1010f2968b
I20260923 22:12:42.668313 6677 ts_tablet_manager.cc:1939] T 3ba7c3c263fa45b2b997ea1010f2968b P b9c412a9d0f84ce98c9a788a73aa1e13: Deleting consensus metadata
I20260923 22:12:42.668781 6172 catalog_manager.cc:4841] TS b9c412a9d0f84ce98c9a788a73aa1e13 (127.1.132.131:44557): tablet 3ba7c3c263fa45b2b997ea1010f2968b (table test-table [id=7b3092a92f3447889cf354f00212abce]) successfully deleted
I20260923 22:12:42.668944 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6510
I20260923 22:12:42.673156 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6157
2026-09-23T22:12:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (775 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 22:12:42.691614 1554 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:42.692873 1554 webserver.cc:460] Webserver started at https://127.0.0.1:33473/ using document root <none> and password file <none>
I20260923 22:12:42.692956 1554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:42.693084 1554 webserver.cc:460] Webserver started at http://127.0.0.1:44927/ using document root <none> and password file <none>
2026-09-23T22:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:42Z Disabled control of system clock
I20260923 22:12:42.700585 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:46835
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:46835
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33473/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/testchainca.cert with env {}
W20260923 22:12:42.782541 6684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.782783 6684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.782806 6684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:42.784248 6684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:42.784310 6684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:42.784322 6684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:42.784334 6684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:42.784345 6684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:42.784358 6684 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:42.784380 6684 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33473/jwks.json
I20260923 22:12:42.786212 6684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:46835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33473/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:42.786496 6684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:42.786731 6684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:42.791673 6692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.791723 6689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:42.791673 6690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:42.791883 6684 server_base.cc:1034] running on GCE node
I20260923 22:12:42.792109 6684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:42.792420 6684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:42.793583 6684 hybrid_clock.cc:648] HybridClock initialized: now 1790201562793556 us; error 42 us; skew 500 ppm
I20260923 22:12:42.794819 6684 webserver.cc:460] Webserver started at http://127.1.132.190:42313/ using document root <none> and password file <none>
I20260923 22:12:42.795063 6684 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:42.795110 6684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:42.795217 6684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:42.796332 6684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "a9de75e0541f487b8368a64b11063b13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "15689895bf3c6ea2fa99d06512e69424"
server_key_iv: "d0d53b489945a8b9b5ef45b84bbda660"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.796660 6684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "a9de75e0541f487b8368a64b11063b13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "15689895bf3c6ea2fa99d06512e69424"
server_key_iv: "d0d53b489945a8b9b5ef45b84bbda660"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.797860 6684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:42.798585 6698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:42.798718 6684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:42.798795 6684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "a9de75e0541f487b8368a64b11063b13"
format_stamp: "Formatted at 2026-09-23 22:12:42 on dist-test-slave-l4q5"
server_key: "15689895bf3c6ea2fa99d06512e69424"
server_key_iv: "d0d53b489945a8b9b5ef45b84bbda660"
server_key_version: "encryptionkey@0"
I20260923 22:12:42.798862 6684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:42.809713 6684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:42.810010 6684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:42.810140 6684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:42.813848 6684 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:46835
I20260923 22:12:42.813884 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:46835 every 8 connection(s)
I20260923 22:12:42.814203 6684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:42.814863 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6684
I20260923 22:12:42.814990 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:42.815140 6751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:42.815181 1554 external_mini_cluster.cc:1428] Setting key 3f42b2bf95164488d0b3fa4f38ccbe0e
I20260923 22:12:42.818359 6751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: Bootstrap starting.
I20260923 22:12:42.818993 6751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:42.819267 6751 log.cc:824] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:42.820035 6751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: No bootstrap required, opened a new log
I20260923 22:12:42.821175 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.815792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.1.132.190:46835)
0923 22:12:42.815993 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.816002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.816035 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:12:42.816547 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.816558 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.816642 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.816791 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:42.816976 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.818299 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.819132 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.819687 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.819818 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:42.820456 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:42.820460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:42.820460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:42.820665 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:42.820706 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:42.820721 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:42.820787 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:42.820864 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:12:42.820946 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260923 22:12:42.822222 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:42.822264 6751 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } }
I20260923 22:12:42.822424 6751 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:42.822459 6751 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9de75e0541f487b8368a64b11063b13, State: Initialized, Role: FOLLOWER
I20260923 22:12:42.822572 6751 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } }
I20260923 22:12:42.822640 6751 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:42.822664 6751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:42.822706 6751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:42.823318 6751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } }
I20260923 22:12:42.823406 6751 leader_election.cc:302] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9de75e0541f487b8368a64b11063b13; no voters:
I20260923 22:12:42.823556 6751 leader_election.cc:288] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:42.823587 6756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:42.823714 6756 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [term 1 LEADER]: Becoming Leader. State: Replica: a9de75e0541f487b8368a64b11063b13, State: Running, Role: LEADER
I20260923 22:12:42.823798 6756 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } }
I20260923 22:12:42.823817 6751 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:42.824213 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9de75e0541f487b8368a64b11063b13. Latest consensus state: current_term: 1 leader_uuid: "a9de75e0541f487b8368a64b11063b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } } }
I20260923 22:12:42.824318 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:42.824165 6757 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9de75e0541f487b8368a64b11063b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9de75e0541f487b8368a64b11063b13" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 46835 } } }
I20260923 22:12:42.824470 6757 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:42.825372 6772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:42.825412 6772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:42.825461 6764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:42.825605 6764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:42.826893 6764 catalog_manager.cc:1347] Generated new cluster ID: 36baf3b22d3144a6a30ac6089fd468be
I20260923 22:12:42.826932 6764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:42.836328 6764 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:42.836835 6764 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:42.845881 6764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9de75e0541f487b8368a64b11063b13: Generated new TSK 0
I20260923 22:12:42.846082 6764 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:12:42.870728 6781 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 22:12:42.870827 6781 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:12:42.871090 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:42.854507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.1.132.190:46835)
0923 22:12:42.854569 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:42.854571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:42.854650 (+ 79us) server_negotiation.cc:407] Connection header received
0923 22:12:42.854739 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:42.854740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:42.854753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:42.854826 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:42.855003 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.855836 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.856700 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.857012 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.857088 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:42.857354 (+ 266us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:42.870821 (+ 13467us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:12:42.870982 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51620: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 22:12:42.871171 6754 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51620: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 22:12:42.871469 6780 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:42.854380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:46835 (local address 127.0.0.1:51620)
0923 22:12:42.854579 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:42.854590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:42.854658 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:42.854865 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:42.854866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:42.854868 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:42.854935 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:42.854937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.855935 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.855936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:42.856624 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:42.856627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:42.857157 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:42.857158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:42.857213 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:42.857238 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:42.871033 (+ 13795us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:12:42.871148 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.132.190:46835: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.132.190:46835: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33473/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 22:12:42.871928 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6684
2026-09-23T22:12:42Z chronyd exiting
I20260923 22:12:42.908031 1554 test_util.cc:175] -----------------------------------------------
I20260923 22:12:42.908120 1554 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201541420007-1554-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (217 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 22:12:42.909133 1554 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:42.909925 1554 webserver.cc:460] Webserver started at https://127.0.0.1:35193/ using document root <none> and password file <none>
I20260923 22:12:42.909976 1554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:42.910118 1554 webserver.cc:460] Webserver started at http://127.0.0.1:41439/ using document root <none> and password file <none>
2026-09-23T22:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:42Z Disabled control of system clock
I20260923 22:12:42.918457 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45333
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:45333
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35193/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/testchainca.cert with env {}
W20260923 22:12:42.998598 6787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:42.998829 6787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:42.998862 6787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:43.000168 6787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:43.000247 6787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:43.000262 6787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:43.000279 6787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:43.000295 6787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:43.000313 6787 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:43.000334 6787 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35193/jwks.json
I20260923 22:12:43.002008 6787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:45333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35193/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:43.002290 6787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:43.002504 6787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:43.007474 6792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.007483 6795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.007565 6793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:43.007910 6787 server_base.cc:1034] running on GCE node
I20260923 22:12:43.008217 6787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:43.008531 6787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:43.009686 6787 hybrid_clock.cc:648] HybridClock initialized: now 1790201563009674 us; error 32 us; skew 500 ppm
I20260923 22:12:43.011113 6787 webserver.cc:460] Webserver started at http://127.1.132.190:46737/ using document root <none> and password file <none>
I20260923 22:12:43.011356 6787 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:43.011417 6787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:43.011554 6787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:43.012683 6787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "f5b4c89dd7284265a2ac2edc96653871"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "4083e3d64a88c44b144839843b752c08"
server_key_iv: "c45f5494311982dbc3472e5ab45a9419"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.013013 6787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "f5b4c89dd7284265a2ac2edc96653871"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "4083e3d64a88c44b144839843b752c08"
server_key_iv: "c45f5494311982dbc3472e5ab45a9419"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.014295 6787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:43.015095 6801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:43.015265 6787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:43.015358 6787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "f5b4c89dd7284265a2ac2edc96653871"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "4083e3d64a88c44b144839843b752c08"
server_key_iv: "c45f5494311982dbc3472e5ab45a9419"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.015422 6787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:43.042585 6787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:43.042928 6787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:43.043067 6787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:43.046929 6787 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:45333
I20260923 22:12:43.046968 6853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:45333 every 8 connection(s)
I20260923 22:12:43.047318 6787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:43.048620 6854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:43.051928 6854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871: Bootstrap starting.
I20260923 22:12:43.052711 6854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:43.052989 6854 log.cc:824] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:43.053409 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6787
I20260923 22:12:43.053517 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:43.053719 1554 external_mini_cluster.cc:1428] Setting key 6aa9c9fc60a2ee613e6213ae115f0622
I20260923 22:12:43.053833 6854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871: No bootstrap required, opened a new log
I20260923 22:12:43.056759 6854 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } }
I20260923 22:12:43.057555 6854 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:43.057683 6854 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4c89dd7284265a2ac2edc96653871, State: Initialized, Role: FOLLOWER
I20260923 22:12:43.057919 6854 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } }
I20260923 22:12:43.058025 6854 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:43.058063 6854 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:43.058111 6854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:43.058923 6854 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } }
I20260923 22:12:43.059039 6854 leader_election.cc:302] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5b4c89dd7284265a2ac2edc96653871; no voters:
I20260923 22:12:43.059219 6854 leader_election.cc:288] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:43.059293 6859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:43.059427 6859 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [term 1 LEADER]: Becoming Leader. State: Replica: f5b4c89dd7284265a2ac2edc96653871, State: Running, Role: LEADER
I20260923 22:12:43.059530 6854 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:43.059553 6859 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } }
I20260923 22:12:43.060065 6861 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5b4c89dd7284265a2ac2edc96653871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } } }
I20260923 22:12:43.060158 6861 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:43.060395 6860 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b4c89dd7284265a2ac2edc96653871. Latest consensus state: current_term: 1 leader_uuid: "f5b4c89dd7284265a2ac2edc96653871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b4c89dd7284265a2ac2edc96653871" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45333 } } }
I20260923 22:12:43.060482 6860 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:43.060606 6863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:43.061136 6863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:43.061647 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.054300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.1.132.190:45333)
0923 22:12:43.054506 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.054514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.054538 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:12:43.055124 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.055132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.055197 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.055325 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:43.055964 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.057213 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.058475 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.059179 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.059330 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.060634 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:43.060643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:43.060644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:43.060885 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:43.060937 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:43.060963 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:43.061300 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:43.061335 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:12:43.061427 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260923 22:12:43.062464 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:43.062764 6863 catalog_manager.cc:1347] Generated new cluster ID: 65857f96e7874e07894c0e9ffcb99cb0
I20260923 22:12:43.062809 6863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:43.072999 6863 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:43.073722 6863 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:43.084215 6863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5b4c89dd7284265a2ac2edc96653871: Generated new TSK 0
I20260923 22:12:43.084426 6863 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:12:43.106880 6884 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 22:12:43.106979 6884 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:12:43.107307 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.093322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.1.132.190:45333)
0923 22:12:43.093368 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.093371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.093461 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:12:43.093534 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.093535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.093548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.093611 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:43.093726 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.094237 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.094998 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.095226 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.095284 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.095446 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:43.107080 (+ 11634us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35193/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:12:43.107238 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35193/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:12:43.107364 6858 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35193/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 22:12:43.107395 6883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:43.093252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:45333 (local address 127.0.0.1:45446)
0923 22:12:43.093403 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:43.093412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:43.093469 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:43.093616 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:43.093617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:43.093618 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:43.093674 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.093676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.094313 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.094314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.094918 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.094920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.095293 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.095294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.095351 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.095356 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:43.107198 (+ 11842us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35193/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:12:43.107304 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.132.190:45333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35193/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260923 22:12:43.107757 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6787
2026-09-23T22:12:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 22:12:43.144550 1554 mini_oidc.cc:152] Starting JWKS server
I20260923 22:12:43.145223 1554 webserver.cc:460] Webserver started at https://127.0.0.1:35875/ using document root <none> and password file <none>
I20260923 22:12:43.145265 1554 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:12:43.145392 1554 webserver.cc:460] Webserver started at http://127.0.0.1:41365/ using document root <none> and password file <none>
2026-09-23T22:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:43Z Disabled control of system clock
I20260923 22:12:43.152479 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:41975
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:44799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:41975
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35875/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:12:43.235706 6890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:43.235922 6890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:43.235944 6890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:43.237123 6890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:43.237169 6890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:43.237179 6890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:43.237190 6890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:43.237201 6890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:43.237212 6890 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:43.237221 6890 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35875/jwks.json
I20260923 22:12:43.238564 6890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:44799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:41975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:41975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35875/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:43.238763 6890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:43.239009 6890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:43.243599 6895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.243635 6896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.243598 6898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:43.243865 6890 server_base.cc:1034] running on GCE node
I20260923 22:12:43.244098 6890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:43.244393 6890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:43.245535 6890 hybrid_clock.cc:648] HybridClock initialized: now 1790201563245488 us; error 69 us; skew 500 ppm
I20260923 22:12:43.246840 6890 webserver.cc:460] Webserver started at http://127.1.132.190:38777/ using document root <none> and password file <none>
I20260923 22:12:43.247074 6890 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:43.247123 6890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:43.247255 6890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:43.248395 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "49a84a1e16b34bd8b2e586e8f803e1ff"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "a5db6d74a5df50764a82336270257831"
server_key_iv: "de4bc7f50b80bfc858e21c269e562fd2"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.248728 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "49a84a1e16b34bd8b2e586e8f803e1ff"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "a5db6d74a5df50764a82336270257831"
server_key_iv: "de4bc7f50b80bfc858e21c269e562fd2"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.249948 6890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:43.250700 6904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:43.250875 6890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:43.250954 6890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "49a84a1e16b34bd8b2e586e8f803e1ff"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "a5db6d74a5df50764a82336270257831"
server_key_iv: "de4bc7f50b80bfc858e21c269e562fd2"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.251024 6890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:43.268925 6890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:43.269215 6890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:43.269313 6890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:43.272960 6890 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:41975
I20260923 22:12:43.272996 6956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:41975 every 8 connection(s)
I20260923 22:12:43.273331 6890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:43.273969 6957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:43.276386 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6890
I20260923 22:12:43.276472 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:43.276615 1554 external_mini_cluster.cc:1428] Setting key 8ff1475e8ff57a5c60a819485a0f521b
I20260923 22:12:43.276661 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff: Bootstrap starting.
I20260923 22:12:43.277336 6957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:43.277598 6957 log.cc:824] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:43.278347 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff: No bootstrap required, opened a new log
I20260923 22:12:43.280637 6957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } }
I20260923 22:12:43.280822 6957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:43.280854 6957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a84a1e16b34bd8b2e586e8f803e1ff, State: Initialized, Role: FOLLOWER
I20260923 22:12:43.280995 6957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } }
I20260923 22:12:43.281076 6957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:43.281116 6957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:43.281158 6957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:43.281915 6957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } }
I20260923 22:12:43.282033 6957 leader_election.cc:302] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49a84a1e16b34bd8b2e586e8f803e1ff; no voters:
I20260923 22:12:43.282200 6957 leader_election.cc:288] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:43.282243 6962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:43.282394 6962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [term 1 LEADER]: Becoming Leader. State: Replica: 49a84a1e16b34bd8b2e586e8f803e1ff, State: Running, Role: LEADER
I20260923 22:12:43.282480 6957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:43.282538 6962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } }
I20260923 22:12:43.282922 6964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49a84a1e16b34bd8b2e586e8f803e1ff. Latest consensus state: current_term: 1 leader_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } } }
I20260923 22:12:43.282944 6963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a84a1e16b34bd8b2e586e8f803e1ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41975 } } }
I20260923 22:12:43.283004 6964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:43.283020 6963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:43.283142 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.277064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.1.132.190:41975)
0923 22:12:43.277233 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.277241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.277265 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:12:43.277781 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.277788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.277858 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.277985 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:43.278202 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.279579 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.280418 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.281024 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.281145 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.281936 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:43.281940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:43.281941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:43.282159 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:43.282203 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:43.282219 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:43.282550 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:43.282606 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:12:43.282697 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260923 22:12:43.283236 6970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:43.283711 6970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:43.284032 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:43.285244 6970 catalog_manager.cc:1347] Generated new cluster ID: 1d41667da4ac425c8d7d386d91657493
I20260923 22:12:43.285296 6970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:43.291260 6970 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:43.291899 6970 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:43.298406 6970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49a84a1e16b34bd8b2e586e8f803e1ff: Generated new TSK 0
I20260923 22:12:43.298633 6970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:43.308991 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.304551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.1.132.190:41975)
0923 22:12:43.304597 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.304599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.304773 (+ 174us) server_negotiation.cc:407] Connection header received
0923 22:12:43.304821 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.304822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.304835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.304925 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:43.305020 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.305799 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.306690 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.307015 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.307083 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.307292 (+ 209us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:43.308601 (+ 1309us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:12:43.308889 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51064: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:43.309060 6960 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51064: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 22:12:43.310701 6986 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:43.304497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:41975 (local address 127.0.0.1:51064)
0923 22:12:43.304702 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:43.304711 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:43.304770 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:43.304924 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:43.304924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:43.304925 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:43.304981 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.304983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.305936 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.305937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.306603 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.306606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.307112 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.307113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.307180 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.307186 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:43.308721 (+ 1535us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:12:43.310600 (+ 1879us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.132.190:41975: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":1687,"thread_start_us":162,"threads_started":1}
I20260923 22:12:43.311110 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6890
E20260923 22:12:43.319710 6987 webserver.cc:533] Webserver: sslize failed:
E20260923 22:12:43.319790 6987 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T22:12:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T22:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:43Z Disabled control of system clock
I20260923 22:12:43.354866 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45661
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:39465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:45661
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/testchainca.cert with env {}
W20260923 22:12:43.431687 6991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:43.431902 6991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:43.431926 6991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:43.433185 6991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:43.433228 6991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:43.433239 6991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:43.433250 6991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:43.433261 6991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:12:43.433271 6991 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:12:43.433282 6991 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 22:12:43.434760 6991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:39465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:45661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:43.434995 6991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:43.435194 6991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:43.439999 6991 server_base.cc:1034] running on GCE node
W20260923 22:12:43.439911 6996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.439930 6997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:43.439920 6999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:43.440460 6991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:43.440797 6991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:43.441978 6991 hybrid_clock.cc:648] HybridClock initialized: now 1790201563441951 us; error 41 us; skew 500 ppm
I20260923 22:12:43.443363 6991 webserver.cc:460] Webserver started at http://127.1.132.190:46819/ using document root <none> and password file <none>
I20260923 22:12:43.443588 6991 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:43.443638 6991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:43.443796 6991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:43.444945 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "b9fe637481f54e65b0c7e628b1ecfeb8"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "1a37b4ee0c824cbbac16f63501198f12"
server_key_iv: "f5a3b58cb02120a1d55775609bb5926e"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.445284 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "b9fe637481f54e65b0c7e628b1ecfeb8"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "1a37b4ee0c824cbbac16f63501198f12"
server_key_iv: "f5a3b58cb02120a1d55775609bb5926e"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.446686 6991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:43.447597 7005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:43.447765 6991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:43.447850 6991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "b9fe637481f54e65b0c7e628b1ecfeb8"
format_stamp: "Formatted at 2026-09-23 22:12:43 on dist-test-slave-l4q5"
server_key: "1a37b4ee0c824cbbac16f63501198f12"
server_key_iv: "f5a3b58cb02120a1d55775609bb5926e"
server_key_version: "encryptionkey@0"
I20260923 22:12:43.447923 6991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:43.461926 6991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:43.462270 6991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:43.462417 6991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:43.466495 6991 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:45661
I20260923 22:12:43.466542 7057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:45661 every 8 connection(s)
I20260923 22:12:43.466876 6991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:43.467878 7058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:43.469218 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 6991
I20260923 22:12:43.469312 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:43.469468 1554 external_mini_cluster.cc:1428] Setting key 301d9ec426a86691863cdc1f2b33a538
I20260923 22:12:43.470527 7058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: Bootstrap starting.
I20260923 22:12:43.471154 7058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:43.471441 7058 log.cc:824] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:43.472249 7058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: No bootstrap required, opened a new log
I20260923 22:12:43.474675 7058 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } }
I20260923 22:12:43.474851 7058 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:43.474887 7058 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9fe637481f54e65b0c7e628b1ecfeb8, State: Initialized, Role: FOLLOWER
I20260923 22:12:43.475013 7058 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } }
I20260923 22:12:43.475086 7058 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:43.475116 7058 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:43.475162 7058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:43.475683 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.470076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.1.132.190:45661)
0923 22:12:43.470271 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.470280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.470306 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:12:43.470795 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.470803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.470874 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.471012 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:43.471279 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.472627 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.473742 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.474192 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.474285 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.475001 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:43.475005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:43.475006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:43.475210 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:43.475254 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:43.475267 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:43.475327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:43.475415 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:12:43.475483 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260923 22:12:43.475870 7058 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } }
I20260923 22:12:43.475972 7058 leader_election.cc:302] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9fe637481f54e65b0c7e628b1ecfeb8; no voters:
I20260923 22:12:43.476141 7058 leader_election.cc:288] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:43.476207 7063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:43.476341 7063 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [term 1 LEADER]: Becoming Leader. State: Replica: b9fe637481f54e65b0c7e628b1ecfeb8, State: Running, Role: LEADER
I20260923 22:12:43.476424 7063 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } }
I20260923 22:12:43.476449 7058 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:43.476719 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } } }
I20260923 22:12:43.476751 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:43.476775 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:43.476878 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9fe637481f54e65b0c7e628b1ecfeb8. Latest consensus state: current_term: 1 leader_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fe637481f54e65b0c7e628b1ecfeb8" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45661 } } }
I20260923 22:12:43.476909 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:43.478132 7078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:43.478479 7078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:43.478560 7067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:43.478744 7067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:43.481323 7067 catalog_manager.cc:1347] Generated new cluster ID: f23b4531042946d7a0631aafb5d899be
I20260923 22:12:43.481374 7067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:43.509084 7067 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:43.509617 7067 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:43.512688 7067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9fe637481f54e65b0c7e628b1ecfeb8: Generated new TSK 0
I20260923 22:12:43.512838 7067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:43.586392 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:43.527165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.1.132.190:45661)
0923 22:12:43.527223 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:43.527226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:43.527332 (+ 106us) server_negotiation.cc:407] Connection header received
0923 22:12:43.527401 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:43.527402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:43.527416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:43.527483 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:12:43.527612 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.528124 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.528899 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.529153 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.529214 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.529373 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:12:43.586142 (+ 56769us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:12:43.586315 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43022: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":31}
W20260923 22:12:43.586476 7061 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43022: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 22:12:43.586485 7087 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:43.527121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:45661 (local address 127.0.0.1:43022)
0923 22:12:43.527264 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:43.527274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:43.527335 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:43.527480 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:43.527481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:43.527483 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:12:43.527552 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.527554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.528220 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.528221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.528823 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:43.528825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:43.529226 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:43.529227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:43.529283 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:43.529288 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:12:43.586281 (+ 56993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:12:43.586400 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.132.190:45661: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260923 22:12:43.586675 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 6991
2026-09-23T22:12:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (255 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:46 dist-test-slave-l4q5 krb5kdc[7099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:46 dist-test-slave-l4q5 krb5kdc[7099](info): set up 2 sockets
Sep 23 22:12:46 dist-test-slave-l4q5 krb5kdc[7099](info): commencing operation
krb5kdc: starting...
W20260923 22:12:47.060308 1554 mini_kdc.cc:121] Time spent starting KDC: real 3.283s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260923 22:12:47.211117 1554 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.151s user 0.000s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260923 22:12:47.326464 1554 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.115s user 0.001s sys 0.000s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260923 22:12:47.542848 1554 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.216s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/test-user.keytab.
W20260923 22:12:47.896118 1554 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.353s user 0.001s sys 0.001s
Sep 23 22:12:47 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:47.943570 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36223
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.034799 7112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.035033 7112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.035056 7112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.036350 7112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:48.036401 7112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.036413 7112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.036424 7112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:48.036437 7112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:48.038035 7112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:48.038301 7112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.038537 7112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:48.043682 7112 server_base.cc:1034] running on GCE node
W20260923 22:12:48.043645 7117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.043645 7118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.043646 7120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.044165 7112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.044484 7112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.045658 7112 hybrid_clock.cc:648] HybridClock initialized: now 1790201568045623 us; error 52 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.048636 7112 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:12:48.049144 7112 webserver.cc:460] Webserver started at http://127.1.132.190:40079/ using document root <none> and password file <none>
I20260923 22:12:48.049352 7112 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.049401 7112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.049526 7112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.050640 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "bd24a1e68dc34963a81e10f8922a7653"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "1a96b8bbf420fef128b9392fb1a65d28"
server_key_iv: "8aa89cb8e5155dc7afddf421aeccd0a4"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.051093 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "bd24a1e68dc34963a81e10f8922a7653"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "1a96b8bbf420fef128b9392fb1a65d28"
server_key_iv: "8aa89cb8e5155dc7afddf421aeccd0a4"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.052522 7112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:48.053709 7127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.054106 7112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:48.054216 7112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "bd24a1e68dc34963a81e10f8922a7653"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "1a96b8bbf420fef128b9392fb1a65d28"
server_key_iv: "8aa89cb8e5155dc7afddf421aeccd0a4"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.054296 7112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.068066 7112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.070252 7112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.070405 7112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.074342 7112 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36223
I20260923 22:12:48.074383 7179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36223 every 8 connection(s)
I20260923 22:12:48.074728 7112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:48.075625 7180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:48.078194 7180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: Bootstrap starting.
I20260923 22:12:48.078851 7180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:48.078941 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7112
I20260923 22:12:48.079052 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:48.079249 1554 external_mini_cluster.cc:1428] Setting key 30bc9291de0ad4db029313059b8c7702
I20260923 22:12:48.079252 7180 log.cc:824] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:48.080238 7180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: No bootstrap required, opened a new log
I20260923 22:12:48.083127 7180 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } }
I20260923 22:12:48.083335 7180 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:48.083379 7180 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd24a1e68dc34963a81e10f8922a7653, State: Initialized, Role: FOLLOWER
I20260923 22:12:48.083523 7180 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } }
I20260923 22:12:48.083599 7180 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:48.083624 7180 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:48.083673 7180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:48.084419 7180 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } }
I20260923 22:12:48.084537 7180 leader_election.cc:302] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd24a1e68dc34963a81e10f8922a7653; no voters:
I20260923 22:12:48.084707 7180 leader_election.cc:288] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:48.084805 7185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:48.084959 7185 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [term 1 LEADER]: Becoming Leader. State: Replica: bd24a1e68dc34963a81e10f8922a7653, State: Running, Role: LEADER
I20260923 22:12:48.084967 7180 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:48.085186 7185 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } }
I20260923 22:12:48.085948 7186 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd24a1e68dc34963a81e10f8922a7653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } } }
I20260923 22:12:48.086057 7186 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:48.085958 7187 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd24a1e68dc34963a81e10f8922a7653. Latest consensus state: current_term: 1 leader_uuid: "bd24a1e68dc34963a81e10f8922a7653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd24a1e68dc34963a81e10f8922a7653" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36223 } } }
I20260923 22:12:48.086242 7187 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653 [sys.catalog]: This master's current role is: LEADER
W20260923 22:12:48.086953 7200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:48.087028 7200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:48.087072 7195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:48.087256 7195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:48.089186 7195 catalog_manager.cc:1347] Generated new cluster ID: 16a11661dc954e33adc4d95ea01cee81
I20260923 22:12:48.089248 7195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:48.091959 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.079829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.1.132.190:36223)
0923 22:12:48.080088 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.080098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.080134 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:12:48.080862 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.080874 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.081032 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.081206 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.081831 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.083248 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.084239 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.084872 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.084986 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.087728 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.087733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.087733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.087748 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.090028 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.090517 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.090521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.090522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.090659 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.090862 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.090863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.090863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.091228 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.091393 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.091479 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.091726 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260923 22:12:48.101478 7195 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:48.102262 7195 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:48.108856 7195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd24a1e68dc34963a81e10f8922a7653: Generated new TSK 0
I20260923 22:12:48.109093 7195 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:12:48.135337 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.224483 7206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.224717 7206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.224743 7206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.225978 7206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.226023 7206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.226048 7206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:48.227603 7206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:48.227841 7206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.228039 7206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.233304 7214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.233299 7211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.233299 7212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.233726 7206 server_base.cc:1034] running on GCE node
I20260923 22:12:48.233960 7206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.234181 7206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.235344 7206 hybrid_clock.cc:648] HybridClock initialized: now 1790201568235327 us; error 31 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.238121 7206 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:12:48.238628 7206 webserver.cc:460] Webserver started at http://127.1.132.129:41513/ using document root <none> and password file <none>
I20260923 22:12:48.238863 7206 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.238911 7206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.239029 7206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.240249 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "b6936753a38d41c193b8ac68848645cc"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "bef0f877e9e090207c8eb180f1d2c243"
server_key_iv: "4cda0e82b4dfedf2fa96b8a026770ee2"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.240600 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "b6936753a38d41c193b8ac68848645cc"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "bef0f877e9e090207c8eb180f1d2c243"
server_key_iv: "4cda0e82b4dfedf2fa96b8a026770ee2"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.242074 7206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:48.242985 7221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.243212 7206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:48.243314 7206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "b6936753a38d41c193b8ac68848645cc"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "bef0f877e9e090207c8eb180f1d2c243"
server_key_iv: "4cda0e82b4dfedf2fa96b8a026770ee2"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.243381 7206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.253345 7206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.255545 7206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.255702 7206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.255928 7206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:48.256276 7206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:48.256313 7206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.256345 7206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:48.256366 7206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.262902 7206 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:40253
I20260923 22:12:48.262981 7334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:40253 every 8 connection(s)
I20260923 22:12:48.263316 7206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:48.270411 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7206
I20260923 22:12:48.270519 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:48.270717 1554 external_mini_cluster.cc:1428] Setting key 94dad25dc3caba0a56a49baadbf8e869
I20260923 22:12:48.273370 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.264177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:40427 (local address 127.1.132.190:36223)
0923 22:12:48.264257 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.264259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.264439 (+ 180us) server_negotiation.cc:407] Connection header received
0923 22:12:48.265452 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.265453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.265465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.265535 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.266420 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.266930 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.267979 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.268244 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.268312 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.270539 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.270541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.270541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.270551 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.271951 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.272415 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.272416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.272417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.272473 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.272731 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.272732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.272733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.272848 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.272903 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.273185 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.273278 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:48.273555 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.264089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36223 (local address 127.1.132.129:40427)
0923 22:12:48.264309 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:48.264355 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:48.265331 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:48.265617 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:48.265622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:48.266024 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:48.266314 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.266320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.267026 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.267027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.267894 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.267896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.268357 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.268358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.268476 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.268938 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:48.268951 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:48.270415 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:48.272052 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.272061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.272064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.272349 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.272565 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.272565 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.272565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.272675 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.272903 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:48.272909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:48.273064 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 22:12:48.273157 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:12:48.273327 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":64,"threads_started":1}
I20260923 22:12:48.273967 7335 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36223
I20260923 22:12:48.274068 7335 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:48.274250 7335 heartbeater.cc:511] Master 127.1.132.190:36223 requested a full tablet report, sending...
I20260923 22:12:48.274776 7144 ts_manager.cc:194] Registered new tserver with Master: b6936753a38d41c193b8ac68848645cc (127.1.132.129:40253)
I20260923 22:12:48.275426 7144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:40427
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:12:48.300109 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.394059 7340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.394318 7340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.394344 7340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.396538 7340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.396646 7340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.396689 7340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:48.400578 7340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:48.400892 7340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.401324 7340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.407791 7345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.407856 7346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.407809 7348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.408311 7340 server_base.cc:1034] running on GCE node
I20260923 22:12:48.408524 7340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.408789 7340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.409968 7340 hybrid_clock.cc:648] HybridClock initialized: now 1790201568409956 us; error 35 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.413089 7340 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:12:48.413589 7340 webserver.cc:460] Webserver started at http://127.1.132.130:41041/ using document root <none> and password file <none>
I20260923 22:12:48.413853 7340 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.413908 7340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.414040 7340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.415190 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "2e691a18d4e448a5ae86ac8a4b8a1697"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "9ab0b8a79afa09e5cd9587bea9bdc7db"
server_key_iv: "ee642e57abe40bd96aab7098c69a714e"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.415537 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "2e691a18d4e448a5ae86ac8a4b8a1697"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "9ab0b8a79afa09e5cd9587bea9bdc7db"
server_key_iv: "ee642e57abe40bd96aab7098c69a714e"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.417019 7340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:48.418036 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.418303 7340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:48.418409 7340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "2e691a18d4e448a5ae86ac8a4b8a1697"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "9ab0b8a79afa09e5cd9587bea9bdc7db"
server_key_iv: "ee642e57abe40bd96aab7098c69a714e"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.418481 7340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.437172 7340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.439282 7340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.439420 7340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.439656 7340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:48.439940 7340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:48.439975 7340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.440002 7340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:48.440018 7340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.445375 7340 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:42455
I20260923 22:12:48.445436 7468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:42455 every 8 connection(s)
I20260923 22:12:48.445739 7340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:48.455750 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.446427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36223 (local address 127.1.132.130:36125)
0923 22:12:48.446641 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:48.446678 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:48.447689 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:48.447925 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:48.447929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:48.448229 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:48.448444 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.448448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.449105 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.449106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.449912 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.449914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.450267 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.450268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.450381 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.450872 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:48.450880 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:48.452296 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:48.454153 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.454161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.454163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.454406 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.454671 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.454672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.454673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.454791 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.455193 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:48.455197 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:48.455328 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 22:12:48.455422 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:12:48.455560 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":141,"threads_started":1}
I20260923 22:12:48.455865 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.446465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:36125 (local address 127.1.132.190:36223)
0923 22:12:48.446519 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.446521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.446764 (+ 243us) server_negotiation.cc:407] Connection header received
0923 22:12:48.447789 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.447791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.447804 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.447856 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.448528 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.449023 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.449998 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.450198 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.450254 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.452409 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.452410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.452411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.452418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.454046 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.454487 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.454488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.454488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.454580 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.454979 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.454980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.454980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.455121 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.455193 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.455537 (+ 344us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.455679 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:48.456032 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7340
I20260923 22:12:48.456120 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:48.456331 1554 external_mini_cluster.cc:1428] Setting key b09a928db0d023cfe7bfad948397edf1
I20260923 22:12:48.456357 7469 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36223
I20260923 22:12:48.456450 7469 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:48.456655 7469 heartbeater.cc:511] Master 127.1.132.190:36223 requested a full tablet report, sending...
I20260923 22:12:48.457175 7144 ts_manager.cc:194] Registered new tserver with Master: 2e691a18d4e448a5ae86ac8a4b8a1697 (127.1.132.130:42455)
I20260923 22:12:48.457554 7144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:36125
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:12:48.487771 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:48.574520 7474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:48.574734 7474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:48.574757 7474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:48.575897 7474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:48.575935 7474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:48.575959 7474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:48.577443 7474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:48.577656 7474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:48.577852 7474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:48.582876 7480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:48.582981 7479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.583094 7474 server_base.cc:1034] running on GCE node
W20260923 22:12:48.582868 7482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:48.583419 7474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:48.583660 7474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:48.584805 7474 hybrid_clock.cc:648] HybridClock initialized: now 1790201568584773 us; error 43 us; skew 500 ppm
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.587460 7474 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:12:48.587895 7474 webserver.cc:460] Webserver started at http://127.1.132.131:41929/ using document root <none> and password file <none>
I20260923 22:12:48.588119 7474 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:48.588163 7474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:48.588320 7474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:48.589501 7474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "0910c431841143a5b2eed66273f85925"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "a931543ca06edea56b97fb7c309407cf"
server_key_iv: "c6b8d1e6ef8bec8e7cc5eb7d33667dec"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.589955 7474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "0910c431841143a5b2eed66273f85925"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "a931543ca06edea56b97fb7c309407cf"
server_key_iv: "c6b8d1e6ef8bec8e7cc5eb7d33667dec"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.591509 7474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:48.592269 7489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.592420 7474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:48.592517 7474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "0910c431841143a5b2eed66273f85925"
format_stamp: "Formatted at 2026-09-23 22:12:48 on dist-test-slave-l4q5"
server_key: "a931543ca06edea56b97fb7c309407cf"
server_key_iv: "c6b8d1e6ef8bec8e7cc5eb7d33667dec"
server_key_version: "encryptionkey@0"
I20260923 22:12:48.592581 7474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:48.603317 7474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:48.605405 7474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:48.605528 7474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:48.605769 7474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:48.606072 7474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:48.606108 7474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.606142 7474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:48.606165 7474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:48.612004 7474 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45139
I20260923 22:12:48.612049 7602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45139 every 8 connection(s)
I20260923 22:12:48.612404 7474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:48.612802 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7474
I20260923 22:12:48.612884 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:48.613065 1554 external_mini_cluster.cc:1428] Setting key 831b7e168a44f48f41bdd1561abe2de5
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:12:48.621626 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.613197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:36065 (local address 127.1.132.190:36223)
0923 22:12:48.613247 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.613248 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.613533 (+ 285us) server_negotiation.cc:407] Connection header received
0923 22:12:48.614641 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.614642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.614652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.614738 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.615330 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.615833 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.616819 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.617003 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.617048 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.619014 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.619015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.619016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.619022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.620274 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.620675 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.620676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.620677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.620724 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.620936 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.620937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.620937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.621066 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.621135 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.621492 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.621556 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:48.621845 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.613191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36223 (local address 127.1.132.131:36065)
0923 22:12:48.613406 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:48.613455 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:48.614478 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:48.614802 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:48.614805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:48.615068 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:48.615263 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.615267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.615897 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.615898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.616747 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:48.616749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.617055 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.617055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:48.617150 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.617653 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:48.617662 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:48.618931 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:48.620352 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.620358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.620360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.620601 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.620769 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:48.620770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:48.620770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:48.620865 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:48.621167 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:48.621173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:48.621343 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:12:48.621434 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:12:48.621618 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260923 22:12:48.622327 7603 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36223
I20260923 22:12:48.622551 7603 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:48.622754 7603 heartbeater.cc:511] Master 127.1.132.190:36223 requested a full tablet report, sending...
I20260923 22:12:48.623131 7144 ts_manager.cc:194] Registered new tserver with Master: 0910c431841143a5b2eed66273f85925 (127.1.132.131:45139)
I20260923 22:12:48.623469 7144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:36065
I20260923 22:12:48.624894 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/test-admin.keytab.
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.641880 1554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 22:12:48.643412 7610 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:48.642712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.1.132.129:40253)
0923 22:12:48.642868 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.642873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.642881 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:48.642904 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.642906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.642918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.642980 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.643334 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44536: BlockingRecv error: recv got EOF from 127.0.0.1:44536 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
W20260923 22:12:48.643460 1554 init.cc:244] Error while opening credential cache '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc)
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.644434 1554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.645257 1554 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:48.653342 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.645857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.1.132.129:40253)
0923 22:12:48.645912 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.645913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.645935 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:12:48.646006 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.646007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.646018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.646064 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.646458 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.647265 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.647981 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.648335 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.648396 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.650516 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.650526 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.650526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.650534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.652050 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.652435 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.652439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.652439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.652497 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.652771 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.652772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.652772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.652934 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.653083 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.653157 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.653231 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260923 22:12:48.655169 7610 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:48.654698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.1.132.129:40253)
0923 22:12:48.654765 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.654767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.654777 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:48.654815 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.654815 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.654825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.654866 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.655127 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44558: BlockingRecv error: recv got EOF from 127.0.0.1:44558 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260923 22:12:48.655262 1554 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.656360 1554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.657291 1554 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:48.664158 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.657856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.1.132.129:40253)
0923 22:12:48.657917 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.657919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.657927 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:48.657988 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.657989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.658003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.658046 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.658442 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.658936 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.659623 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.659813 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.659867 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.661861 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.661862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.661863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.661870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.663157 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.663471 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.663473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.663473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.663556 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.663717 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.663718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.663719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.663812 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.663893 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.663988 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.664067 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260923 22:12:48.665642 7610 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:48.665089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.1.132.129:40253)
0923 22:12:48.665127 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.665128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.665156 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:12:48.665213 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.665214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.665222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.665262 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.665598 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44584: BlockingRecv error: recv got EOF from 127.0.0.1:44584 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:48.665676 1554 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201541420007-1554-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.666620 1554 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:48.667387 1554 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:12:48.673792 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:48.667903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.1.132.129:40253)
0923 22:12:48.667952 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:48.667953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:48.668004 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:12:48.668053 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:48.668054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:48.668061 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:48.668102 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:48.668442 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.668923 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.669623 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:48.669807 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:48.669869 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:48.671695 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:48.671696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:48.671697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:48.671703 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:48.672985 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.673253 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.673253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.673254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.673297 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:48.673414 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:48.673415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:48.673415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:48.673531 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:48.673615 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:48.673671 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:12:48.673727 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:12:48.674441 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7206
I20260923 22:12:48.678665 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7340
I20260923 22:12:48.682619 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7474
I20260923 22:12:48.686432 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7112
2026-09-23T22:12:48Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4927 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7623](info): set up 2 sockets
Sep 23 22:12:48 dist-test-slave-l4q5 krb5kdc[7623](info): commencing operation
krb5kdc: starting...
W20260923 22:12:49.652370 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:49 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab.
Entry for principal oryx/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab.
I20260923 22:12:49.771876 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:32829
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab
--principal=oryx/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:32829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:49.851671 7636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:49.851905 7636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:49.851939 7636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:49.853197 7636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:49.853250 7636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:49.853268 7636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:49.853284 7636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:49.853299 7636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:49.854847 7636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:32829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab
--principal=oryx/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:49.855130 7636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:49.855348 7636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:49.860051 7636 server_base.cc:1034] running on GCE node
W20260923 22:12:49.859961 7644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.859972 7641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:49.859951 7642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:49.860610 7636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:49.860905 7636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:49.862061 7636 hybrid_clock.cc:648] HybridClock initialized: now 1790201569862046 us; error 29 us; skew 500 ppm
Sep 23 22:12:49 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:49.885967 7636 init.cc:377] Logged in from keytab as oryx/127.1.132.190@KRBTEST.COM (short username oryx)
I20260923 22:12:49.886471 7636 webserver.cc:460] Webserver started at http://127.1.132.190:43639/ using document root <none> and password file <none>
I20260923 22:12:49.886703 7636 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:49.886750 7636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:49.886858 7636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:49.887959 7636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "a2c970d8fc8d411f81767475089d8ac3"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-l4q5"
server_key: "87aeebbd7486bff2186c608f43ebfabf"
server_key_iv: "9694a692e8151a7d6a12e3eac84165fc"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.888315 7636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "a2c970d8fc8d411f81767475089d8ac3"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-l4q5"
server_key: "87aeebbd7486bff2186c608f43ebfabf"
server_key_iv: "9694a692e8151a7d6a12e3eac84165fc"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.889647 7636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:49.890408 7651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:49.890638 7636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:49.890718 7636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "a2c970d8fc8d411f81767475089d8ac3"
format_stamp: "Formatted at 2026-09-23 22:12:49 on dist-test-slave-l4q5"
server_key: "87aeebbd7486bff2186c608f43ebfabf"
server_key_iv: "9694a692e8151a7d6a12e3eac84165fc"
server_key_version: "encryptionkey@0"
I20260923 22:12:49.890795 7636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:49.898651 7636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:49.900825 7636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:49.900946 7636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:49.904789 7636 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:32829
I20260923 22:12:49.904824 7703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:32829 every 8 connection(s)
I20260923 22:12:49.905165 7636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:49.906173 7704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:49.906822 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7636
I20260923 22:12:49.906910 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:49.907088 1554 external_mini_cluster.cc:1428] Setting key ad84c1975eac95d832464aa569c1d095
I20260923 22:12:49.908938 7704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Bootstrap starting.
I20260923 22:12:49.909633 7704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:49.909951 7704 log.cc:824] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:49.910737 7704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: No bootstrap required, opened a new log
I20260923 22:12:49.913087 7704 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } }
I20260923 22:12:49.913246 7704 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:49.913269 7704 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c970d8fc8d411f81767475089d8ac3, State: Initialized, Role: FOLLOWER
I20260923 22:12:49.913401 7704 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } }
I20260923 22:12:49.913492 7704 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:49.913522 7704 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:49.913576 7704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:12:49 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:49.914229 7704 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } }
I20260923 22:12:49.914338 7704 leader_election.cc:302] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2c970d8fc8d411f81767475089d8ac3; no voters:
I20260923 22:12:49.914510 7704 leader_election.cc:288] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:49.914551 7709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:49.914657 7709 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 LEADER]: Becoming Leader. State: Replica: a2c970d8fc8d411f81767475089d8ac3, State: Running, Role: LEADER
I20260923 22:12:49.914922 7709 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } }
I20260923 22:12:49.914919 7704 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:49.915395 7710 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c970d8fc8d411f81767475089d8ac3. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } }
I20260923 22:12:49.916000 7710 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.915413 7711 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } }
I20260923 22:12:49.916224 7711 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:49.916466 7723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:49.917042 7723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:49.919023 7723 catalog_manager.cc:1347] Generated new cluster ID: 995fa1ca495d450cb7f5403b5d89b1b6
I20260923 22:12:49.919078 7723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:49.919358 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:49.907600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.1.132.190:32829)
0923 22:12:49.907770 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:49.907778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:49.907803 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:12:49.908252 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:49.908260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:49.908318 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:49.908441 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:49.909079 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.910493 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.911375 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:49.912031 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:49.912155 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:49.914877 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:49.914882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:49.914882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:49.914896 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:49.917336 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.917719 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.917722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.917723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.917824 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:49.918088 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:49.918089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:49.918090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:49.918414 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:49.918549 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:49.919006 (+ 457us) server_negotiation.cc:299] Negotiation successful
0923 22:12:49.919145 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260923 22:12:49.935735 7723 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:49.936398 7723 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:49.938278 7723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Generated new TSK 0
I20260923 22:12:49.938503 7723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
I20260923 22:12:49.971807 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:50.055131 7730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:50.055373 7730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:50.055398 7730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:50.056921 7730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:50.056975 7730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:50.057014 7730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:50.058604 7730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:50.058892 7730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:50.059113 7730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.064335 7736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.064468 7738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.064365 7730 server_base.cc:1034] running on GCE node
W20260923 22:12:50.064365 7735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.064741 7730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.064970 7730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.066141 7730 hybrid_clock.cc:648] HybridClock initialized: now 1790201570066127 us; error 30 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.069459 7730 init.cc:377] Logged in from keytab as oryx/127.1.132.129@KRBTEST.COM (short username oryx)
I20260923 22:12:50.069957 7730 webserver.cc:460] Webserver started at http://127.1.132.129:36577/ using document root <none> and password file <none>
I20260923 22:12:50.070168 7730 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.070223 7730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.070346 7730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:50.071391 7730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "4ae9d80cb9b24908a79c464afc2351f6"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "9d541c2f23bd46e8e5eafa710c07dfbd"
server_key_iv: "facdf455509d1ba54be2a4207d528540"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.071728 7730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "4ae9d80cb9b24908a79c464afc2351f6"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "9d541c2f23bd46e8e5eafa710c07dfbd"
server_key_iv: "facdf455509d1ba54be2a4207d528540"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.073127 7730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:50.073961 7745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.074187 7730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:50.074282 7730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "4ae9d80cb9b24908a79c464afc2351f6"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "9d541c2f23bd46e8e5eafa710c07dfbd"
server_key_iv: "facdf455509d1ba54be2a4207d528540"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.074347 7730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.081095 7730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.083262 7730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.083405 7730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.083623 7730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:50.084010 7730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:50.084065 7730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.084098 7730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:50.084116 7730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.090158 7730 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:33001
I20260923 22:12:50.090234 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:33001 every 8 connection(s)
I20260923 22:12:50.090525 7730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:50.096535 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7730
I20260923 22:12:50.096627 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:50.096809 1554 external_mini_cluster.cc:1428] Setting key b77e360509976cc2cfc0d05b262df597
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:50.100930 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.091290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:35467 (local address 127.1.132.190:32829)
0923 22:12:50.091386 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.091388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.091635 (+ 247us) server_negotiation.cc:407] Connection header received
0923 22:12:50.092116 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.092117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.092129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.092207 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.093192 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.093748 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.094756 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.094978 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.095033 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.097826 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.097827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.097827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.097835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.099354 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.099842 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.099843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.099843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.099897 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.100239 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.100240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.100240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.100352 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.100400 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.100765 (+ 365us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.100859 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 22:12:50.101119 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.091208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:35467)
0923 22:12:50.091438 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.091479 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.091990 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.092282 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.092286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.092879 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.093092 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.093097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.093843 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.093844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.094680 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.094683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.095059 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.095060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.095179 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.095810 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.095818 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.097730 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.099465 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.099473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.099475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.099764 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.100007 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.100008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.100008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.100149 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.100405 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.100410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.100585 (+ 175us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.100691 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.100869 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260923 22:12:50.101761 7859 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:50.101871 7859 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.102172 7859 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:50.102733 7668 ts_manager.cc:194] Registered new tserver with Master: 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:50.103531 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:35467
WARNING: no policy specified for oryx/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
I20260923 22:12:50.123986 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:50.204100 7864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:50.204361 7864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:50.204385 7864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:50.205588 7864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:50.205631 7864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:50.205657 7864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:50.207139 7864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:50.207369 7864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:50.207589 7864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.212538 7870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.212603 7869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.212684 7864 server_base.cc:1034] running on GCE node
W20260923 22:12:50.212544 7872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.212980 7864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.213235 7864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.214380 7864 hybrid_clock.cc:648] HybridClock initialized: now 1790201570214367 us; error 27 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.217121 7864 init.cc:377] Logged in from keytab as oryx/127.1.132.130@KRBTEST.COM (short username oryx)
I20260923 22:12:50.217566 7864 webserver.cc:460] Webserver started at http://127.1.132.130:46359/ using document root <none> and password file <none>
I20260923 22:12:50.217780 7864 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.217823 7864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.217947 7864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:50.219053 7864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "16eaefabd0f1431bb4beafd5468c84cc"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "95823692310a70b586cdd3f365f298ac"
server_key_iv: "c8a3c9e3cb5871869d0ec8d10605ab2f"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.219381 7864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "16eaefabd0f1431bb4beafd5468c84cc"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "95823692310a70b586cdd3f365f298ac"
server_key_iv: "c8a3c9e3cb5871869d0ec8d10605ab2f"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.220798 7864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:50.221597 7879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.221778 7864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:50.221866 7864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "16eaefabd0f1431bb4beafd5468c84cc"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "95823692310a70b586cdd3f365f298ac"
server_key_iv: "c8a3c9e3cb5871869d0ec8d10605ab2f"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.221931 7864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.235631 7864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.237852 7864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.238001 7864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.238229 7864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:50.238528 7864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:50.238590 7864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.238621 7864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:50.238646 7864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.244310 7864 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39467
I20260923 22:12:50.244361 7992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39467 every 8 connection(s)
I20260923 22:12:50.244696 7864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:50.248934 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7864
I20260923 22:12:50.249056 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:50.249248 1554 external_mini_cluster.cc:1428] Setting key bfa81cb81b205a9face7f9d94fd8b286
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:50.254922 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.245445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:43027 (local address 127.1.132.190:32829)
0923 22:12:50.245539 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.245541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.245753 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:12:50.246239 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.246240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.246250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.246303 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.247242 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.247753 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.248808 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.249102 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.249193 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.251459 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.251461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.251461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.251469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.253264 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.253786 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.253787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.253788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.253855 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.254191 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.254192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.254193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.254320 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.254384 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.254760 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.254852 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:12:50.255142 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.245416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:43027)
0923 22:12:50.245623 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.245662 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.246135 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.246367 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.246370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.246940 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.247147 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.247152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.247837 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.247838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.248691 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.248694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.249187 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.249188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.249316 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.249841 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.249850 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.251312 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.253372 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.253383 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.253386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.253695 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.253925 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.253926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.253927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.254076 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.254391 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.254399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.254586 (+ 187us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.254711 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.254897 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260923 22:12:50.255683 7993 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:50.255805 7993 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.256062 7993 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:50.256526 7668 ts_manager.cc:194] Registered new tserver with Master: 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467)
I20260923 22:12:50.256965 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:43027
WARNING: no policy specified for oryx/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
I20260923 22:12:50.275542 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:50.357472 7998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:50.357697 7998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:50.357720 7998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:50.358891 7998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:50.358942 7998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:50.358968 7998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:50.360591 7998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:32829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:50.360862 7998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:50.361121 7998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.366508 8006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.366495 8003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.366621 7998 server_base.cc:1034] running on GCE node
W20260923 22:12:50.366492 8004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.366983 7998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.367254 7998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.368396 7998 hybrid_clock.cc:648] HybridClock initialized: now 1790201570368386 us; error 31 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.370793 7998 init.cc:377] Logged in from keytab as oryx/127.1.132.131@KRBTEST.COM (short username oryx)
I20260923 22:12:50.371227 7998 webserver.cc:460] Webserver started at http://127.1.132.131:40163/ using document root <none> and password file <none>
I20260923 22:12:50.371435 7998 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.371484 7998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.371608 7998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:50.372747 7998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "2820bab0ef3e40b28f8da7fcf4640e20"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb7932e65889c73da2ebc96d7732d6d5"
server_key_iv: "4b7bb90b50add5f2276bed6c656d399c"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.373072 7998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "2820bab0ef3e40b28f8da7fcf4640e20"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb7932e65889c73da2ebc96d7732d6d5"
server_key_iv: "4b7bb90b50add5f2276bed6c656d399c"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.374346 7998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:50.375108 8013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.375262 7998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:50.375336 7998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "2820bab0ef3e40b28f8da7fcf4640e20"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb7932e65889c73da2ebc96d7732d6d5"
server_key_iv: "4b7bb90b50add5f2276bed6c656d399c"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.375399 7998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.389132 7998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.391208 7998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.391350 7998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.391567 7998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:50.391867 7998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:50.391901 7998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.391929 7998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:50.391947 7998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.397517 7998 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:41423
I20260923 22:12:50.397557 8126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:41423 every 8 connection(s)
I20260923 22:12:50.397878 7998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:50.400478 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 7998
I20260923 22:12:50.400571 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:50.400730 1554 external_mini_cluster.cc:1428] Setting key d15318cc72a3ed1788c1e3475d18fcff
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:50.406947 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.398600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59253 (local address 127.1.132.190:32829)
0923 22:12:50.398651 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.398653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.398803 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:12:50.399276 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.399276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.399287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.399349 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.400283 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.400867 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.402121 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.402319 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.402376 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.404548 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.404549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.404550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.404557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.405878 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.406228 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.406229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.406229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.406278 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.406461 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.406461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.406461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.406559 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.406604 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.406822 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.406880 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:12:50.407169 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.398532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:59253)
0923 22:12:50.398691 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.398734 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.399191 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.399402 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.399406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.399988 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.400207 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.400212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.400943 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.400944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.402045 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.402048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.402412 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.402414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.402575 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.403054 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.403062 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.404449 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.405950 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.405955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.405957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.406166 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.406324 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.406325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.406325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.406419 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.406604 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.406606 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.406727 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.406804 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.406954 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260923 22:12:50.407606 8127 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:50.407713 8127 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:50.407884 8127 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:50.408265 7668 ts_manager.cc:194] Registered new tserver with Master: 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131:41423)
I20260923 22:12:50.408600 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:59253
I20260923 22:12:50.412724 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.189.keytab.
Entry for principal oryx/127.1.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.189.keytab.
I20260923 22:12:50.439253 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:45827
--webserver_interface=127.1.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.189.keytab
--principal=oryx/127.1.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:32829,127.1.132.189:45827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:50.520990 8132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:50.521202 8132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:50.521225 8132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:50.522411 8132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:50.522461 8132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:50.522475 8132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:50.522485 8132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:50.522504 8132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:50.524016 8132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:32829,127.1.132.189:45827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.189.keytab
--principal=oryx/127.1.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:45827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
W20260923 22:12:50.524127 8132 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.132.190:32829,127.1.132.189:45827'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 22:12:50.524315 8132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:50.524554 8132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.529436 8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.529495 8132 server_base.cc:1034] running on GCE node
W20260923 22:12:50.529441 8138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.529441 8137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.529966 8132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.530292 8132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.531463 8132 hybrid_clock.cc:648] HybridClock initialized: now 1790201570531444 us; error 33 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.534160 8132 init.cc:377] Logged in from keytab as oryx/127.1.132.189@KRBTEST.COM (short username oryx)
I20260923 22:12:50.534624 8132 webserver.cc:460] Webserver started at http://127.1.132.189:40475/ using document root <none> and password file <none>
I20260923 22:12:50.534839 8132 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.534888 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.535015 8132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:50.536100 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/instance:
uuid: "5ecd4c221abd498d845cd89ed3e15f84"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "30543e1e2e66686337b890bb773caefd"
server_key_iv: "0adb0aa8a1c430801d886691cc2d05be"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.536490 8132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal/instance:
uuid: "5ecd4c221abd498d845cd89ed3e15f84"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "30543e1e2e66686337b890bb773caefd"
server_key_iv: "0adb0aa8a1c430801d886691cc2d05be"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.537899 8132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:50.538672 8147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.538820 8132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:50.538910 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "5ecd4c221abd498d845cd89ed3e15f84"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "30543e1e2e66686337b890bb773caefd"
server_key_iv: "0adb0aa8a1c430801d886691cc2d05be"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.538972 8132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.552598 8132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.554708 8132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.554843 8132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.558687 8132 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.189:45827
I20260923 22:12:50.558770 8199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.189:45827 every 8 connection(s)
I20260923 22:12:50.559082 8132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:50.560113 8200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:50.561774 8200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:50.563970 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 8132
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.189@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
I20260923 22:12:50.572283 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.562759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.1.132.190:32829)
0923 22:12:50.562880 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.562883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.563043 (+ 160us) server_negotiation.cc:407] Connection header received
0923 22:12:50.563558 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.563560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.563573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.563636 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.564330 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.564966 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.566308 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.566513 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.566706 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.569455 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.569456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.569457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.569467 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.570839 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.571289 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.571291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.571291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.571347 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.571594 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.571596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.571596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.571755 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.571813 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.572082 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.572167 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 22:12:50.572467 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.562690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36796)
0923 22:12:50.562921 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.562961 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.563456 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.563712 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.563715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.563978 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.564246 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.564250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.565063 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.565063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.566189 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.566191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.566612 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.566613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.566740 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.567705 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.567716 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.569354 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.570906 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.570914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.570917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.571218 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.571409 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.571409 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.571410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.571530 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.571817 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.571822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.571961 (+ 139us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.572053 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.572231 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":101,"threads_started":1}
I20260923 22:12:50.573570 8200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:50.574558 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.564618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.1.132.189:45827)
0923 22:12:50.564813 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.564817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.564833 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:12:50.564852 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.564854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.564876 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.564944 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.565520 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.566492 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.568225 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.568677 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.568740 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.571260 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.571269 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.571270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.571277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.572848 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.573580 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.573582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.573582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.573641 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.573917 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.573918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.573918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.574138 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.574282 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.574386 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.574460 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.189@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
I20260923 22:12:50.581830 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.573986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.1.132.189:45827)
0923 22:12:50.574147 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.574148 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.574154 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:50.574174 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.574175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.574184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.574250 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.574601 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.575309 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.575932 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.576166 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.576293 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.578727 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.578728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.578728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.578735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.580721 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.581046 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.581047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.581047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.581097 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.581245 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.581246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.581246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.581354 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.581409 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.581593 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.581692 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260923 22:12:50.581907 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.573948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49434)
0923 22:12:50.573982 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.573991 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.574018 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.574272 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.574273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.574415 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.574467 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.574468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.575360 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.575361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.575846 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.575848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.576384 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.576387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.576503 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.577085 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.577090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.578613 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.580822 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.580823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.580824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.580974 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.581149 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.581150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.581150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.581197 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.581405 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.581406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.581443 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.581569 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.581796 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":127}
I20260923 22:12:50.583421 8200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Bootstrap starting.
I20260923 22:12:50.584021 8200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:50.584414 8200 log.cc:824] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:50.585597 8200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: No bootstrap required, opened a new log
I20260923 22:12:50.587306 8200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } }
I20260923 22:12:50.587461 8200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:50.587483 8200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ecd4c221abd498d845cd89ed3e15f84, State: Initialized, Role: FOLLOWER
I20260923 22:12:50.587831 8200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } }
I20260923 22:12:50.588137 8207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } } }
I20260923 22:12:50.588215 8200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:50.588263 8207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:50.590291 8218 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:50.590404 8218 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:50.592768 8132 master_runner.cc:428] Detected that this master 5ecd4c221abd498d845cd89ed3e15f84 is joining an existing cluster
I20260923 22:12:50.592841 8132 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.132.189:45827
I20260923 22:12:50.599870 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.593965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49450)
0923 22:12:50.594181 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.594189 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.594247 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.594344 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.594344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.594560 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.594627 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.594629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.595264 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.595264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.595813 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.595815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.596164 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.596165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.596281 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.596788 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.596793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.597233 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.598826 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.598827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.598828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.599024 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.599194 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.599195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.599195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.599243 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.599469 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.599469 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.599519 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.599591 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.599740 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260923 22:12:50.599923 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.593962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.1.132.189:45827)
0923 22:12:50.594037 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.594039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.594231 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:12:50.594291 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.594291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.594300 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.594341 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.594695 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.595191 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.595883 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.596087 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.596142 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.597354 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.597355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.597356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.597364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.598745 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.599088 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.599089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.599089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.599142 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.599291 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.599292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.599292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.599405 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.599470 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.599616 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.599826 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":55}
I20260923 22:12:50.601022 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.594084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36806 (local address 127.1.132.190:32829)
0923 22:12:50.594137 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.594139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.594166 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:50.594243 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.594244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.594256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.594314 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.594815 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.595578 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.596740 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.597018 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.597096 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.598044 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.598045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.598046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.598053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.599557 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.600285 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.600286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.600286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.600351 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.600540 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.600541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.600541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.600653 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.600708 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.600853 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.600924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:50.601042 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.593848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36806)
0923 22:12:50.594108 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.594119 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.594177 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.594365 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.594366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.594619 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.594711 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.594713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.595687 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.595688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.596546 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.596549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.597111 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.597112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.597188 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.597551 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.597555 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.597941 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.599646 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.599647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.599647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.599889 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.600421 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.600422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.600422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.600478 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.600706 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.600707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.600764 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.600822 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.600925 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.190@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
I20260923 22:12:50.612362 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.603104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49462)
0923 22:12:50.603248 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.603299 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.603378 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.603518 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.603522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.603886 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.603952 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.603955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.604653 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.604654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.605617 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.605620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.605957 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.605957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.606046 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.606607 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.606612 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.608287 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.610624 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.610635 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.610637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.610866 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.611134 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.611135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.611135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.611325 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.611761 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.611770 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.611828 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.612087 (+ 259us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.612269 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260923 22:12:50.612419 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.603180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.1.132.189:45827)
0923 22:12:50.603221 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.603223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.603381 (+ 158us) server_negotiation.cc:407] Connection header received
0923 22:12:50.603453 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.603454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.603464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.603513 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.604045 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.604579 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.605700 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.605885 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.605948 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.608373 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.608375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.608375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.608383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.610496 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.610961 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.610963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.610963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.611034 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.611395 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.611396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.611396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.611556 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.611632 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.612224 (+ 592us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.612321 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:50.613121 7668 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.132.189:45827: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84"
member_type: NON_VOTER
last_known_addr {
host: "127.1.132.189"
port: 45827
}
attrs {
promote: true
}
}
dest_uuid: "a2c970d8fc8d411f81767475089d8ac3"
cas_config_opid_index: -1
I20260923 22:12:50.613440 7668 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } }
I20260923 22:12:50.613937 8228 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } }
I20260923 22:12:50.614235 8174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:50.615132 8228 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.132.189:45827
I20260923 22:12:50.615237 8174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 22:12:50.615329 8228 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } }
I20260923 22:12:50.615393 8228 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.615389 8132 master.cc:537] Master@127.1.132.189:45827 shutting down...
I20260923 22:12:50.615614 8228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:12:50.616880 7655 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 22:12:50.617337 7655 consensus_peers.cc:487] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 -> Peer 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189:45827): Couldn't send request to peer 5ecd4c221abd498d845cd89ed3e15f84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:50.618117 8132 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:50.618208 8132 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:50.618259 8132 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: stopping tablet replica
I20260923 22:12:50.631327 8132 master.cc:552] Master@127.1.132.189:45827 shutdown complete.
I20260923 22:12:50.631397 8132 master_runner.cc:305] Clearing existing system tablet
I20260923 22:12:50.631526 8132 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.631572 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.632337 8132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:12:50.632860 8232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.632983 8132 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.633038 8132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.633061 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "5ecd4c221abd498d845cd89ed3e15f84"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "30543e1e2e66686337b890bb773caefd"
server_key_iv: "0adb0aa8a1c430801d886691cc2d05be"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.633671 8132 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:50.635142 8132 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:12:50.635236 8132 log.cc:1196] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 22:12:50.635404 8132 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Deleting consensus metadata
I20260923 22:12:50.635507 8132 master_runner.cc:315] Copying system tablet from 127.1.132.190:32829
I20260923 22:12:50.636073 8132 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: tablet copy: Beginning tablet copy session from remote peer at address 127.1.132.190:32829
I20260923 22:12:50.643326 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.636485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.1.132.190:32829)
0923 22:12:50.636552 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.636554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.636632 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:12:50.636720 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.636720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.636730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.636779 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.637181 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.637681 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.638836 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.639050 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.639110 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.640300 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.640302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.640302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.640311 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.642006 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.642452 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.642453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.642454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.642518 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.642750 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.642751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.642752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.642882 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.642948 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.643179 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.643256 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:50.643469 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.636405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36814)
0923 22:12:50.636560 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.636572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.636649 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.636797 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.636797 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.636997 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.637067 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.637069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.637782 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.637783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.638715 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.638719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.639150 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.639151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.639227 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.639740 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.639746 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.640158 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.642103 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.642104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.642105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.642351 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.642584 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.642584 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.642585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.642652 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.642971 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.642973 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.643031 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.643136 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.643273 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260923 22:12:50.643847 7688 tablet_copy_service.cc:140] P a2c970d8fc8d411f81767475089d8ac3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5ecd4c221abd498d845cd89ed3e15f84 ({username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:36814)
I20260923 22:12:50.644114 7688 tablet_copy_service.cc:161] P a2c970d8fc8d411f81767475089d8ac3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5ecd4c221abd498d845cd89ed3e15f84 at {username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:36814: session id = 5ecd4c221abd498d845cd89ed3e15f84-00000000000000000000000000000000
I20260923 22:12:50.646027 7688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:12:50.647125 8132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:12:50.648176 8132 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: tablet copy: Starting download of 0 data blocks...
I20260923 22:12:50.648368 8132 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: tablet copy: Starting download of 1 WAL segments...
I20260923 22:12:50.649749 8132 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:12:50.650933 7688 tablet_copy_service.cc:342] P a2c970d8fc8d411f81767475089d8ac3: Request end of tablet copy session 5ecd4c221abd498d845cd89ed3e15f84-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:36814
I20260923 22:12:50.651019 7688 tablet_copy_service.cc:433] P a2c970d8fc8d411f81767475089d8ac3: ending tablet copy session 5ecd4c221abd498d845cd89ed3e15f84-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5ecd4c221abd498d845cd89ed3e15f84
I20260923 22:12:50.654193 8132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.655757 8239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.655849 8240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.655855 8242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.656366 8132 server_base.cc:1034] running on GCE node
I20260923 22:12:50.656507 8132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.656734 8132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.657922 8132 hybrid_clock.cc:648] HybridClock initialized: now 1790201570657913 us; error 29 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.659587 8132 init.cc:377] Logged in from keytab as oryx/127.1.132.189@KRBTEST.COM (short username oryx)
I20260923 22:12:50.659864 8132 webserver.cc:460] Webserver started at http://127.1.132.189:40305/ using document root <none> and password file <none>
I20260923 22:12:50.659972 8132 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.660013 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.660645 8132 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.661144 8249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.661340 8132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.661386 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "5ecd4c221abd498d845cd89ed3e15f84"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "30543e1e2e66686337b890bb773caefd"
server_key_iv: "0adb0aa8a1c430801d886691cc2d05be"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.661449 8132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.673949 8132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.675805 8132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.678750 8132 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.189:45827
I20260923 22:12:50.679005 8301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.189:45827 every 8 connection(s)
I20260923 22:12:50.679101 8132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:50.680019 8302 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:12:50.680598 8302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Bootstrap starting.
I20260923 22:12:50.683651 8302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:12:50.683928 8302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Bootstrap complete.
I20260923 22:12:50.684151 8302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } }
I20260923 22:12:50.684278 8302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5ecd4c221abd498d845cd89ed3e15f84, State: Initialized, Role: LEARNER
I20260923 22:12:50.684327 8302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } }
I20260923 22:12:50.684513 8305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } }
I20260923 22:12:50.684599 8305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:50.684619 8302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:50.686233 8316 catalog_manager.cc:1259] Loaded cluster ID: 995fa1ca495d450cb7f5403b5d89b1b6
I20260923 22:12:50.686290 8316 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: loading cluster ID for follower catalog manager: success
I20260923 22:12:50.687155 8316 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: acquiring CA information for follower catalog manager: success
I20260923 22:12:50.687407 8316 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:50.733135 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.726755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.1.132.189:45827)
0923 22:12:50.726896 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.726897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.726905 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:50.726929 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.726930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.726939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.726995 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.727565 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.728275 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.728976 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.729170 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.729226 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.730485 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.730486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.730486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.730493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.732089 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.732424 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.732426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.732426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.732479 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.732646 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.732647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.732647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.732750 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.732834 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.732957 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.733037 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.1.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.188.keytab.
Entry for principal oryx/127.1.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.188.keytab.
I20260923 22:12:50.760427 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:41703
--webserver_interface=127.1.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.188.keytab
--principal=oryx/127.1.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:50.846233 8320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:50.846477 8320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:50.846499 8320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:50.847922 8320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:50.848004 8320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:50.848019 8320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:50.848030 8320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:50.848045 8320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:50.849788 8320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.188.keytab
--principal=oryx/127.1.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:41703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:50.850020 8320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:50.850214 8320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:50.855167 8326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.855191 8325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:50.855183 8328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:50.855480 8320 server_base.cc:1034] running on GCE node
I20260923 22:12:50.855754 8320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:50.856046 8320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:50.857234 8320 hybrid_clock.cc:648] HybridClock initialized: now 1790201570857214 us; error 31 us; skew 500 ppm
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:50.859809 8320 init.cc:377] Logged in from keytab as oryx/127.1.132.188@KRBTEST.COM (short username oryx)
I20260923 22:12:50.860455 8320 webserver.cc:460] Webserver started at http://127.1.132.188:44733/ using document root <none> and password file <none>
I20260923 22:12:50.860690 8320 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.860747 8320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.860857 8320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:50.862007 8320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/instance:
uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb6c0832686ff10037db2c94dc1773b8"
server_key_iv: "5dc310409eae7b3a923d9dceace93795"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.862332 8320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal/instance:
uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb6c0832686ff10037db2c94dc1773b8"
server_key_iv: "5dc310409eae7b3a923d9dceace93795"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.863662 8320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:50.864421 8335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.864643 8320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:50.864740 8320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb6c0832686ff10037db2c94dc1773b8"
server_key_iv: "5dc310409eae7b3a923d9dceace93795"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.864805 8320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.874989 8320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:50.877270 8320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:50.877401 8320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:50.881822 8320 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.188:41703
I20260923 22:12:50.881865 8387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.188:41703 every 8 connection(s)
I20260923 22:12:50.882210 8320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:50.883211 8388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:50.885460 8388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:50.885798 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 8320
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.188@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
I20260923 22:12:50.895756 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.886277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36828)
0923 22:12:50.886617 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.886655 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.887152 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.887388 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.887390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.887810 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.888030 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.888036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.888826 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.888827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.889724 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.889726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.890126 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.890127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.890202 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.890893 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.890902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.892485 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.894076 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.894085 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.894088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.894372 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.894572 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.894572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.894573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.894729 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.895014 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.895019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.895221 (+ 202us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.895344 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.895565 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":92,"threads_started":1}
I20260923 22:12:50.895766 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.886514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.1.132.190:32829)
0923 22:12:50.886567 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.886568 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.886743 (+ 175us) server_negotiation.cc:407] Connection header received
0923 22:12:50.887278 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.887279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.887291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.887362 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.888165 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.888722 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.889816 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.890027 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.890087 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.892620 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.892621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.892621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.892629 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.893995 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.894437 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.894439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.894439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.894506 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.894816 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.894817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.894817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.894948 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.895005 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.895413 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.895515 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:50.896322 8388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:50.896874 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.886493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.1.132.188:41703)
0923 22:12:50.886625 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.886635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.886669 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:12:50.887139 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.887149 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.887176 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.887266 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.887911 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.888891 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.889684 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.890094 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.890202 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.893199 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.893208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.893209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.893217 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.895423 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.896025 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.896029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.896029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.896092 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.896280 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.896281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.896281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.896421 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.896576 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.896687 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.896773 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.188@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
I20260923 22:12:50.904004 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.896643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.1.132.189:45827)
0923 22:12:50.896697 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.896698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.896705 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:50.896730 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.896731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.896744 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.896797 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.897272 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.897807 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.898533 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.898861 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.898922 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.901074 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.901075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.901075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.901082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.902451 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.902820 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.902821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.902822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.902882 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.903144 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.903145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.903145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.903281 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.903346 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.903849 (+ 503us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.903935 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:50.904007 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.896580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49498)
0923 22:12:50.896628 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.896638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.896668 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.896812 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.896812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.897126 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.897177 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.897179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.897905 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.897906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.898408 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.898409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.898998 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.898999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.899061 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.899624 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.899628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.900936 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.902548 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.902549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.902550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.902733 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.902968 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.902969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.902970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.903041 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.903381 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.903382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.903551 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.903818 (+ 267us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.903935 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:12:50.904508 8388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201570, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.188@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
I20260923 22:12:50.912859 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.904797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.0.0.1:38678)
0923 22:12:50.904845 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.904855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.904890 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.904988 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.904988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.905137 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.905183 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.905185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.905829 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.905830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.906573 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.906576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.907136 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.907137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.907223 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.907660 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.907666 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.909065 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.911354 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.911355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.911356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.911583 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.911961 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.911962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.911962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.912030 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.912586 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.912587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.912622 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.912686 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.912786 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:12:50.912933 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.904848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.1.132.188:41703)
0923 22:12:50.904895 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.904897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.904906 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:50.904927 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.904928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.904936 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.904983 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.905273 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.905758 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.906791 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.907039 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.907102 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.909224 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.909227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.909227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.909238 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.911230 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.911753 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.911755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.911756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.911815 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.912233 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.912234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.912235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.912373 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.912582 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.912742 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.912839 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":30}
I20260923 22:12:50.914606 8388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Bootstrap starting.
I20260923 22:12:50.915423 8388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:50.915694 8388 log.cc:824] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:50.916481 8388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: No bootstrap required, opened a new log
I20260923 22:12:50.917961 8388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } }
I20260923 22:12:50.918104 8388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:50.918128 8388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba692dfa53543938b11f1a3f9ab89e0, State: Initialized, Role: FOLLOWER
I20260923 22:12:50.918255 8388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } }
I20260923 22:12:50.918525 8393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } } }
I20260923 22:12:50.918569 8388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:50.918613 8393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:50.920158 8404 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:50.920240 8404 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:50.922540 8320 master_runner.cc:428] Detected that this master 4ba692dfa53543938b11f1a3f9ab89e0 is joining an existing cluster
I20260923 22:12:50.922591 8320 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.132.188:41703
I20260923 22:12:50.930433 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.1.132.188:41703)
0923 22:12:50.923910 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.923911 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.924000 (+ 89us) server_negotiation.cc:407] Connection header received
0923 22:12:50.924093 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.924093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.924102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.924127 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.924605 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925295 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926561 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926849 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926887 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.928068 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.928070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.928070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.928076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.929383 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.929766 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.929767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.929767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.929824 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.930016 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.930017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.930017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.930113 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.930158 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.930300 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.930368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:12:50.930866 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.0.0.1:38688)
0923 22:12:50.923942 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.923952 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.924030 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.924204 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.924205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.924460 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.924522 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.924525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925382 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.925382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.926473 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.926476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926954 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.927019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.927430 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.927437 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.927966 (+ 529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.929460 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.929461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.929462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.929672 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.929895 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.929896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.929896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.929964 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.930163 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.930164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.930211 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.930283 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.930437 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":45,"thread_start_us":60,"threads_started":1}
I20260923 22:12:50.944027 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.1.132.189:45827)
0923 22:12:50.923606 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.923608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.923817 (+ 209us) server_negotiation.cc:407] Connection header received
0923 22:12:50.923998 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.923999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.924008 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.924036 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.924528 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925302 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926477 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926780 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926821 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.941421 (+ 14600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.941423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.941423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.941431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.942778 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943103 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.943104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.943105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.943156 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943361 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.943363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.943363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.943528 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.943643 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.943843 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.943934 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:50.944064 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49508)
0923 22:12:50.923751 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.923759 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.923928 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.924097 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.924098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.924341 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.924431 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.924433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925410 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.925411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.926383 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.926385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926916 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.926980 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.940878 (+ 13898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.940882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.941275 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.942867 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.942868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.942869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.943039 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.943212 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.943213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.943272 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.943676 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.943678 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.943738 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.943809 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.943946 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":85,"thread_start_us":63,"threads_started":1}
I20260923 22:12:50.944303 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.1.132.190:32829)
0923 22:12:50.923640 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.923642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.923789 (+ 147us) server_negotiation.cc:407] Connection header received
0923 22:12:50.923888 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.923889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.923900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.923930 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.924521 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925302 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926477 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926780 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926826 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.941407 (+ 14581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.941409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.941409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.941420 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.943185 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943522 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.943523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.943523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.943578 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943738 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.943738 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.943739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.943874 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.943940 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.944109 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.944218 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:12:50.944607 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.923574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36836)
0923 22:12:50.923716 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.923726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.923814 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.923994 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.923995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.924314 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.924431 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.924433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.925410 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.925411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.926383 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.926385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.926918 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.926918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.926980 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.940729 (+ 13749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.940735 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.941276 (+ 541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.943283 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.943284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.943457 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.943629 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.943629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.943629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.943688 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.943966 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.943967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.944008 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.944379 (+ 371us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.944501 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":28,"thread_start_us":90,"threads_started":1}
I20260923 22:12:50.948256 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7730
Sep 23 22:12:50 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201569, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.190@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
I20260923 22:12:50.954005 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.946645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.0.0.1:38698)
0923 22:12:50.946692 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.946704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.946746 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.946901 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.946901 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.947051 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.947091 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.947093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.947840 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.947841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.948657 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.948661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.949132 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.949134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.949212 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.949700 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.949706 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.951213 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.952980 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.952982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.952983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.953195 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.953375 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.953375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.953376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.953436 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.953685 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.953686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.953735 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.953798 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.953906 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:12:50.954005 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.946751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.1.132.188:41703)
0923 22:12:50.946797 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.946799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.946805 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:50.946837 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.946838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.946847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.946892 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.947152 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.947640 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.948789 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.949059 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.949122 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.951308 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.951310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.951311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.951318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.952884 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.953258 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.953259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.953260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.953318 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.953490 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.953491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.953491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.953628 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.953697 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.953829 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.953910 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:12:50.954937 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:33001
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:32829
--webserver_port=36577
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:50.954932 7668 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.132.188:41703: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
member_type: NON_VOTER
last_known_addr {
host: "127.1.132.188"
port: 41703
}
attrs {
promote: true
}
}
dest_uuid: "a2c970d8fc8d411f81767475089d8ac3"
cas_config_opid_index: 5
I20260923 22:12:50.955142 7668 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } }
I20260923 22:12:50.955448 8228 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.956251 8228 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.132.188:41703
I20260923 22:12:50.956324 8228 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.956384 8228 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.958158 8362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:50.959187 8362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:12:50.959442 8320 master.cc:537] Master@127.1.132.188:41703 shutting down...
I20260923 22:12:50.960408 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.955546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49520)
0923 22:12:50.955633 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.955644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.955682 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.956709 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.956711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.956712 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:50.956758 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.956760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.957780 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.957782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.959032 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.959036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.960079 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.960080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.960160 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.960224 (+ 64us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.960278 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.960329 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 22:12:50.960594 8227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:50.960948 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.956460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49520 (local address 127.1.132.189:45827)
0923 22:12:50.956518 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.956520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.956529 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:50.956555 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.956556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.956578 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.956648 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:50.956891 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.957677 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.959207 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.959798 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.959867 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.959941 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.960831 (+ 890us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.960885 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:50.961275 8276 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 LEARNER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:12:50.961572 8227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260923 22:12:50.962060 7653 consensus_peers.cc:487] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 -> Peer 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703): Couldn't send request to peer 4ba692dfa53543938b11f1a3f9ab89e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:50.962232 8276 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.963764 8305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c970d8fc8d411f81767475089d8ac3. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.963852 8305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:50.963889 8305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: NON_VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: true } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.963927 8305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:50.964540 8320 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:50.964718 8320 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:50.964769 8320 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: stopping tablet replica
I20260923 22:12:50.965322 8227 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: attempting to promote NON_VOTER 5ecd4c221abd498d845cd89ed3e15f84 to VOTER
I20260923 22:12:50.965565 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } }
I20260923 22:12:50.966110 8276 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 LEARNER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260923 22:12:50.966380 7653 consensus_peers.cc:487] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 -> Peer 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703): Couldn't send request to peer 4ba692dfa53543938b11f1a3f9ab89e0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:50.966430 8228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260923 22:12:50.967801 8227 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.968931 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.969022 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.969067 8228 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.969118 8228 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:50.969367 8276 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.970434 8415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:50.970517 8415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:50.979777 8320 master.cc:552] Master@127.1.132.188:41703 shutdown complete.
I20260923 22:12:50.979843 8320 master_runner.cc:305] Clearing existing system tablet
I20260923 22:12:50.979969 8320 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:50.980010 8320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:50.980767 8320 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.981288 8420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.981401 8320 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:50.981447 8320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:50.981468 8320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb6c0832686ff10037db2c94dc1773b8"
server_key_iv: "5dc310409eae7b3a923d9dceace93795"
server_key_version: "encryptionkey@0"
I20260923 22:12:50.982268 8320 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:50.983980 8320 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:12:50.984057 8320 log.cc:1196] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 22:12:50.984277 8320 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Deleting consensus metadata
I20260923 22:12:50.984392 8320 master_runner.cc:315] Copying system tablet from 127.1.132.190:32829
I20260923 22:12:50.985019 8320 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: tablet copy: Beginning tablet copy session from remote peer at address 127.1.132.190:32829
I20260923 22:12:50.995476 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.985409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.1.132.190:32829)
0923 22:12:50.985461 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:50.985464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:50.988425 (+ 2961us) server_negotiation.cc:407] Connection header received
0923 22:12:50.988488 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:50.988489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:50.988503 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:50.988574 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:50.988966 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.989747 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.990953 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.991252 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.991320 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.992495 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:50.992497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:50.992497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:50.992505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:50.994322 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.994719 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.994720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.994721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.994782 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:50.994963 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:50.994963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:50.994963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:50.995094 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:50.995157 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:50.995303 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 22:12:50.995382 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:12:50.995584 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:50.985348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:36838)
0923 22:12:50.988320 (+ 2972us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:50.988335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:50.988428 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:50.988611 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:50.988611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:50.988822 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:50.988882 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.988884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.989903 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.989905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.990852 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:50.990855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:50.991333 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:50.991334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:50.991394 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:50.991933 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:50.991939 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:50.992406 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:50.994409 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.994410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.994411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.994641 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.994840 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:50.994841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:50.994841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:50.994906 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:50.995154 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:50.995155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:50.995206 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:12:50.995274 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:12:50.995388 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"thread_start_us":64,"threads_started":1}
I20260923 22:12:50.996114 7688 tablet_copy_service.cc:140] P a2c970d8fc8d411f81767475089d8ac3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4ba692dfa53543938b11f1a3f9ab89e0 ({username='oryx', principal='oryx/127.1.132.188@KRBTEST.COM'} at 127.0.0.1:36838)
I20260923 22:12:50.996225 7688 tablet_copy_service.cc:161] P a2c970d8fc8d411f81767475089d8ac3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4ba692dfa53543938b11f1a3f9ab89e0 at {username='oryx', principal='oryx/127.1.132.188@KRBTEST.COM'} at 127.0.0.1:36838: session id = 4ba692dfa53543938b11f1a3f9ab89e0-00000000000000000000000000000000
I20260923 22:12:50.996798 7688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:12:50.997270 8320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:12:50.998631 8320 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: tablet copy: Starting download of 0 data blocks...
I20260923 22:12:50.998760 8320 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: tablet copy: Starting download of 1 WAL segments...
I20260923 22:12:50.999541 8320 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:12:51.000444 7688 tablet_copy_service.cc:342] P a2c970d8fc8d411f81767475089d8ac3: Request end of tablet copy session 4ba692dfa53543938b11f1a3f9ab89e0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.132.188@KRBTEST.COM'} at 127.0.0.1:36838
I20260923 22:12:51.000494 7688 tablet_copy_service.cc:433] P a2c970d8fc8d411f81767475089d8ac3: ending tablet copy session 4ba692dfa53543938b11f1a3f9ab89e0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4ba692dfa53543938b11f1a3f9ab89e0
I20260923 22:12:51.003098 8320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:51.004338 8430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.004551 8427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:51.004588 8320 server_base.cc:1034] running on GCE node
W20260923 22:12:51.008389 8428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:51.008603 8320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:51.008771 8320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:51.012010 8320 hybrid_clock.cc:648] HybridClock initialized: now 1790201571011985 us; error 50 us; skew 500 ppm
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:51.013561 8320 init.cc:377] Logged in from keytab as oryx/127.1.132.188@KRBTEST.COM (short username oryx)
I20260923 22:12:51.013830 8320 webserver.cc:460] Webserver started at http://127.1.132.188:36911/ using document root <none> and password file <none>
I20260923 22:12:51.013950 8320 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:51.013983 8320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:51.014727 8320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:51.015372 8437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.015540 8320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:51.015601 8320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb6c0832686ff10037db2c94dc1773b8"
server_key_iv: "5dc310409eae7b3a923d9dceace93795"
server_key_version: "encryptionkey@0"
I20260923 22:12:51.015686 8320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:51.028257 8320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:51.030068 8320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:51.032975 8320 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.188:41703
I20260923 22:12:51.033200 8489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.188:41703 every 8 connection(s)
I20260923 22:12:51.033324 8320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:51.034148 8490 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:12:51.034720 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Bootstrap starting.
I20260923 22:12:51.037804 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:12:51.038143 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: Bootstrap complete.
I20260923 22:12:51.038431 8490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } }
I20260923 22:12:51.038605 8490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4ba692dfa53543938b11f1a3f9ab89e0, State: Initialized, Role: LEARNER
I20260923 22:12:51.038681 8490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } }
I20260923 22:12:51.038938 8490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:51.038975 8493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:51.039093 8493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:12:51.044543 8504 catalog_manager.cc:1259] Loaded cluster ID: 995fa1ca495d450cb7f5403b5d89b1b6
I20260923 22:12:51.044611 8504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: loading cluster ID for follower catalog manager: success
I20260923 22:12:51.045637 8504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: acquiring CA information for follower catalog manager: success
I20260923 22:12:51.045950 8504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 22:12:51.055860 8413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:51.056098 8413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:51.056133 8413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:51.057471 8413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:51.057523 8413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:51.057562 8413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:51.059469 8413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:33001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=36577
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:51.059754 8413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:51.059963 8413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:51.065264 8509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.065258 8512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.065258 8510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:51.066090 8413 server_base.cc:1034] running on GCE node
I20260923 22:12:51.066318 8413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:51.066586 8413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:51.067749 8413 hybrid_clock.cc:648] HybridClock initialized: now 1790201571067735 us; error 34 us; skew 500 ppm
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:51.070317 8413 init.cc:377] Logged in from keytab as oryx/127.1.132.129@KRBTEST.COM (short username oryx)
I20260923 22:12:51.070734 8413 webserver.cc:460] Webserver started at http://127.1.132.129:36577/ using document root <none> and password file <none>
I20260923 22:12:51.070930 8413 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:51.070993 8413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:51.072458 8413 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.073048 8519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.073241 8413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:51.073344 8413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "4ae9d80cb9b24908a79c464afc2351f6"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "9d541c2f23bd46e8e5eafa710c07dfbd"
server_key_iv: "facdf455509d1ba54be2a4207d528540"
server_key_version: "encryptionkey@0"
I20260923 22:12:51.073697 8413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:51.085668 8413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:51.087918 8413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:51.088096 8413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:51.088359 8413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:51.088675 8413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:51.088707 8413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.088737 8413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:51.088760 8413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.094899 8413 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:33001
I20260923 22:12:51.094997 8632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:33001 every 8 connection(s)
I20260923 22:12:51.095281 8413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:51.100692 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 8413
I20260923 22:12:51.100859 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7864
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
I20260923 22:12:51.107259 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.096037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:35265 (local address 127.1.132.188:41703)
0923 22:12:51.096143 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.096145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.096329 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:51.097457 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.097458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.097467 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.097524 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.098201 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.099189 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.099969 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100294 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100359 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.103864 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.103866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.103867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.103875 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.105590 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106120 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.106121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.106121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.106180 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106477 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.106478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.106478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.106637 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.106677 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.107061 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.107156 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
I20260923 22:12:51.107652 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.095978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:36461 (local address 127.1.132.189:45827)
0923 22:12:51.096039 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.096041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.096282 (+ 241us) server_negotiation.cc:407] Connection header received
0923 22:12:51.097339 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.097340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.097353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.097382 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.098217 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.098987 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100239 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100483 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100543 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.104665 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.104667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.104667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.104675 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.106440 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106827 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.106828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.106828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.106882 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.107053 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.107053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.107054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.107183 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.107222 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.107474 (+ 252us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.107554 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:51.107837 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.095983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.1.132.129:35265)
0923 22:12:51.096266 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.096276 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.097236 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.097542 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.097543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.097893 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.098108 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.098113 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.099252 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.099253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.099886 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.099888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100367 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.100509 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.101382 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.101386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.103756 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.105679 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.105688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.105691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.106027 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.106252 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.106253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.106401 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.106726 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.106733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.106912 (+ 179us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.107361 (+ 449us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.107589 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260923 22:12:51.107955 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.095983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.1.132.129:36461)
0923 22:12:51.096138 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.096204 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.097213 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.097494 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.097498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.097920 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.098110 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.098116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.099071 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.099072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.099921 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.099922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100596 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.100666 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.101402 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.101406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.104557 (+ 3151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.106520 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.106521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.106750 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.106945 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.106945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.106995 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.107261 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.107262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.107303 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.107763 (+ 460us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.107858 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260923 22:12:51.108430 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:39467
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:32829
--webserver_port=46359
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:51.108501 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.096448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:40443 (local address 127.1.132.190:32829)
0923 22:12:51.096520 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.096522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.096531 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.097338 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.097339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.097351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.097406 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.098221 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.098721 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100027 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100345 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100398 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.104658 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.104660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.104660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.104668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.106580 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106946 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.106947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.106948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.107017 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.107212 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.107212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.107213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.107333 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.107374 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.108312 (+ 938us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.108418 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:12:51.108503 8635 heartbeater.cc:347] Connected to a master server at 127.1.132.189:45827
I20260923 22:12:51.108623 8635 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.108793 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.096287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:40443)
0923 22:12:51.096391 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.096400 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.097230 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.097494 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.097498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.097902 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.098110 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.098116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.098817 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.098817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.099918 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.099921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.100446 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.100447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.100526 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.101358 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.101368 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.104545 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.106660 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.106661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.106662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.106863 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.107085 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.107086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.107086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.107148 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.107666 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.107666 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.108142 (+ 476us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.108609 (+ 467us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.108709 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":25,"thread_start_us":53,"threads_started":1}
I20260923 22:12:51.108809 8635 heartbeater.cc:511] Master 127.1.132.189:45827 requested a full tablet report, sending...
I20260923 22:12:51.109349 8266 ts_manager.cc:194] Registered new tserver with Master: 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:51.109880 8633 heartbeater.cc:347] Connected to a master server at 127.1.132.188:41703
I20260923 22:12:51.109934 8633 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.110008 8633 heartbeater.cc:511] Master 127.1.132.188:41703 requested a full tablet report, sending...
I20260923 22:12:51.110116 8634 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:51.110157 8634 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.110209 8634 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:51.110347 8450 ts_manager.cc:194] Registered new tserver with Master: 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:51.110481 7668 ts_manager.cc:194] Re-registered known tserver with Master: 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:51.110870 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:40443
W20260923 22:12:51.195874 8641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:51.196101 8641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:51.196135 8641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:51.197584 8641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:51.197669 8641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:51.197712 8641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:51.199391 8641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:39467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=46359
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:51.199692 8641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:51.199937 8641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:51.205080 8647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.205257 8646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.205363 8649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:51.205299 8641 server_base.cc:1034] running on GCE node
I20260923 22:12:51.205565 8641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:51.205842 8641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:51.207031 8641 hybrid_clock.cc:648] HybridClock initialized: now 1790201571207012 us; error 34 us; skew 500 ppm
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:51.210065 8641 init.cc:377] Logged in from keytab as oryx/127.1.132.130@KRBTEST.COM (short username oryx)
I20260923 22:12:51.210621 8641 webserver.cc:460] Webserver started at http://127.1.132.130:46359/ using document root <none> and password file <none>
I20260923 22:12:51.210887 8641 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:51.210973 8641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:51.212867 8641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:51.213596 8656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.213862 8641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.213980 8641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "16eaefabd0f1431bb4beafd5468c84cc"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "95823692310a70b586cdd3f365f298ac"
server_key_iv: "c8a3c9e3cb5871869d0ec8d10605ab2f"
server_key_version: "encryptionkey@0"
I20260923 22:12:51.214418 8641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:51.251367 8641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:51.254318 8641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:51.254472 8641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:51.254720 8641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:51.255059 8641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:51.255095 8641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.255126 8641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:51.255149 8641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.262389 8641 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39467
I20260923 22:12:51.262440 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39467 every 8 connection(s)
I20260923 22:12:51.262902 8641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:51.264473 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 8641
I20260923 22:12:51.264707 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7998
I20260923 22:12:51.269817 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:41423
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:32829
--webserver_port=40163
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:51.277777 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.264010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:53191 (local address 127.1.132.190:32829)
0923 22:12:51.264053 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.264055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.264220 (+ 165us) server_negotiation.cc:407] Connection header received
0923 22:12:51.268441 (+ 4221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.268442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.268457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.268516 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.268927 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269640 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.270731 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.271018 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.271061 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.274096 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.274097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.274098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.274107 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.276006 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276562 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.276563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.276564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.276628 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276954 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.276954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.276955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.277119 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.277165 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.277582 (+ 417us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.277675 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:51.277777 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.263628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:35431 (local address 127.1.132.189:45827)
0923 22:12:51.263675 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.263677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.263950 (+ 273us) server_negotiation.cc:407] Connection header received
0923 22:12:51.265808 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.265809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.265822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.265938 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.267048 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.267823 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269142 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269385 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269424 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.273665 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.273668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.273668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.273713 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.275871 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276564 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.276565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.276565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.276628 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276953 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.276954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.276955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.277096 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.277150 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.277582 (+ 432us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.277675 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:51.277881 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.263687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:40051 (local address 127.1.132.188:41703)
0923 22:12:51.263734 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.263736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.263950 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:12:51.265702 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.265704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.265721 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.265751 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.267047 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.267569 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269148 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269469 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269507 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.274149 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.274152 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.274152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.274161 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.276033 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276626 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.276628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.276628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.276704 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.277014 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.277015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.277016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.277159 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.277211 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.277577 (+ 366us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.277781 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:51.278383 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.263928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:53191)
0923 22:12:51.264144 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.264152 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.265659 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.268575 (+ 2916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.268577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.268770 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.268831 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.268833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269723 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.270626 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.270629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.271145 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.271146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.271207 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.271674 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.271683 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.273987 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.276086 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276467 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.276696 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276876 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.277195 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.277195 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.277395 (+ 200us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.277872 (+ 477us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.278112 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":5,"thread_start_us":74,"threads_started":1}
I20260923 22:12:51.278385 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.263649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.1.132.130:40051)
0923 22:12:51.263819 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.263862 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.265562 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.266008 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.266014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.266562 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.266930 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.266937 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.267790 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.267791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.269045 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.269047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269578 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.269681 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.270431 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.270436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.274028 (+ 3592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.276141 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276494 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.276776 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276936 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.277245 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.277246 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.277395 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.278054 (+ 659us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.278173 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":57,"thread_start_us":99,"threads_started":1}
I20260923 22:12:51.279034 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.263649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.1.132.130:35431)
0923 22:12:51.263894 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.263903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.265561 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.266015 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.266016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.266569 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.266934 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.266937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.267918 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.267919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.269047 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.269049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.269510 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.269510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.269680 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.270337 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.270347 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.273542 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.275987 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276002 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276467 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.276696 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.276697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.276697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.276876 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.277184 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.277192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.277395 (+ 203us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.278429 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.278933 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":77,"thread_start_us":91,"threads_started":1}
I20260923 22:12:51.279124 8771 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:51.279223 8771 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.279345 8770 heartbeater.cc:347] Connected to a master server at 127.1.132.188:41703
I20260923 22:12:51.279379 8770 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.279431 8771 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:51.279433 8770 heartbeater.cc:511] Master 127.1.132.188:41703 requested a full tablet report, sending...
I20260923 22:12:51.279618 8772 heartbeater.cc:347] Connected to a master server at 127.1.132.189:45827
I20260923 22:12:51.279652 8772 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.279703 8772 heartbeater.cc:511] Master 127.1.132.189:45827 requested a full tablet report, sending...
I20260923 22:12:51.279875 7668 ts_manager.cc:194] Re-registered known tserver with Master: 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467)
I20260923 22:12:51.279946 8266 ts_manager.cc:194] Registered new tserver with Master: 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467)
I20260923 22:12:51.280393 8450 ts_manager.cc:194] Registered new tserver with Master: 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467)
I20260923 22:12:51.280498 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:53191
W20260923 22:12:51.361452 8777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:51.361698 8777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:51.361737 8777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:51.363327 8777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:51.363404 8777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:51.363451 8777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:51.365276 8777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:41423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=40163
--tserver_master_addrs=127.1.132.190:32829,127.1.132.189:45827,127.1.132.188:41703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:51.365559 8777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:51.365818 8777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:51.370929 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.370931 8783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:51.371079 8782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:51.371205 8777 server_base.cc:1034] running on GCE node
I20260923 22:12:51.371410 8777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:51.371683 8777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:51.372871 8777 hybrid_clock.cc:648] HybridClock initialized: now 1790201571372850 us; error 27 us; skew 500 ppm
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:51.375613 8777 init.cc:377] Logged in from keytab as oryx/127.1.132.131@KRBTEST.COM (short username oryx)
I20260923 22:12:51.376089 8777 webserver.cc:460] Webserver started at http://127.1.132.131:40163/ using document root <none> and password file <none>
I20260923 22:12:51.376333 8777 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:51.376389 8777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:51.377859 8777 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.378499 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.378660 8777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:51.378746 8777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "2820bab0ef3e40b28f8da7fcf4640e20"
format_stamp: "Formatted at 2026-09-23 22:12:50 on dist-test-slave-l4q5"
server_key: "fb7932e65889c73da2ebc96d7732d6d5"
server_key_iv: "4b7bb90b50add5f2276bed6c656d399c"
server_key_version: "encryptionkey@0"
I20260923 22:12:51.379103 8777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:51.410761 8777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:51.412595 8777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:51.412714 8777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:51.412940 8777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:12:51.413228 8777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:51.413259 8777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.413290 8777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:51.413312 8777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:51.418939 8777 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:41423
I20260923 22:12:51.418987 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:41423 every 8 connection(s)
I20260923 22:12:51.419320 8777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:51.426694 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 8777
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.188@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.189@KRBTEST.COM
Sep 23 22:12:51 dist-test-slave-l4q5 krb5kdc[7623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201571, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:51.432379 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:53595 (local address 127.1.132.188:41703)
0923 22:12:51.420533 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.420535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.421706 (+ 1171us) server_negotiation.cc:407] Connection header received
0923 22:12:51.421733 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.421734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.421744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.421793 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.422399 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423174 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.424925 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425208 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425260 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.429475 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.429476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.429477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.429484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.430959 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.431349 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.431350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.431350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.431402 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.431688 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.431688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.431689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.431815 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.431851 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.432208 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.432287 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:51.432585 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.1.132.131:53595)
0923 22:12:51.421630 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.421635 (+ 5us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.421670 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.421810 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.421811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.422080 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.422314 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.422319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423938 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.423939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.424848 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.424850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425269 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.425345 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.425965 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.425975 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.429391 (+ 3416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.431061 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.431062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.431063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.431281 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.431466 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.431467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.431467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.431612 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.431887 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.431893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.432050 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.432156 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.432358 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"mutex_wait_us":37,"thread_start_us":55,"threads_started":1}
I20260923 22:12:51.433115 8906 heartbeater.cc:347] Connected to a master server at 127.1.132.188:41703
I20260923 22:12:51.433215 8906 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.433409 8906 heartbeater.cc:511] Master 127.1.132.188:41703 requested a full tablet report, sending...
I20260923 22:12:51.433835 8450 ts_manager.cc:194] Registered new tserver with Master: 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131:41423)
I20260923 22:12:51.434100 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:60243)
0923 22:12:51.421527 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.421535 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.421575 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.421743 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.421744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.422090 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.422316 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.422321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423673 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.423674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.424741 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.424744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425205 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.425345 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.426037 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.426042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.428824 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.432846 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.432847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.432848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.433036 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.433517 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.433518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.433567 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.433819 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.433820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.433855 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.433904 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.433990 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1387,"mutex_wait_us":70,"thread_start_us":105,"threads_started":1}
I20260923 22:12:51.434104 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.1.132.131:59139)
0923 22:12:51.420297 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.420342 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.421268 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.421623 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.421628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.422090 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:51.422315 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.422321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423797 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.423797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.424873 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.424874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425275 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.425345 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.426124 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:51.426128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:51.428474 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:51.430641 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.430651 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.430654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.430961 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.433274 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:51.433275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:51.433332 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:51.433609 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:51.433610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:51.433649 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.433701 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.433814 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":31,"thread_start_us":115,"threads_started":1}
I20260923 22:12:51.434340 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59139 (local address 127.1.132.189:45827)
0923 22:12:51.420154 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.420156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.420473 (+ 317us) server_negotiation.cc:407] Connection header received
0923 22:12:51.421385 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.421386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.421396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.421467 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.423263 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423723 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.424966 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425165 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425223 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.428571 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.428573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.428573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.428582 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.430541 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433147 (+ 2606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.433148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.433149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.433204 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433388 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.433389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.433389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.433519 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.433584 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.434174 (+ 590us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.434258 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:51.434517 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.420129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:60243 (local address 127.1.132.190:32829)
0923 22:12:51.420232 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.420234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.421594 (+ 1360us) server_negotiation.cc:407] Connection header received
0923 22:12:51.421621 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.421623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.421634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.421695 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.422426 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.423168 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.424828 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.425125 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.425191 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.428912 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.428913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.428914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.428921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.432731 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433406 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.433408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.433408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.433455 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.433627 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.433628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.433628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.433745 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.433795 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.434385 (+ 590us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.434460 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:12:51.434790 8908 heartbeater.cc:347] Connected to a master server at 127.1.132.189:45827
I20260923 22:12:51.434841 8908 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.434896 8908 heartbeater.cc:511] Master 127.1.132.189:45827 requested a full tablet report, sending...
I20260923 22:12:51.435040 8907 heartbeater.cc:347] Connected to a master server at 127.1.132.190:32829
I20260923 22:12:51.435091 8907 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:51.435150 8907 heartbeater.cc:511] Master 127.1.132.190:32829 requested a full tablet report, sending...
I20260923 22:12:51.435168 8266 ts_manager.cc:194] Registered new tserver with Master: 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131:41423)
I20260923 22:12:51.435349 7668 ts_manager.cc:194] Re-registered known tserver with Master: 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131:41423)
I20260923 22:12:51.435828 7668 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:60243
I20260923 22:12:51.437111 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.427810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.1.132.190:32829)
0923 22:12:51.428059 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.428062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.428276 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:12:51.428391 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.428393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.428418 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.429221 (+ 803us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.429262 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.430091 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.431172 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.431462 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.431775 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.433077 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.433079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.433079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.433086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.435575 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.435989 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.435990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.435990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.436066 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.436313 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.436313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.436314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.436433 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.436886 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.436925 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.437017 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260923 22:12:51.438002 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.428065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.1.132.188:41703)
0923 22:12:51.428412 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.428414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.428499 (+ 85us) server_negotiation.cc:407] Connection header received
0923 22:12:51.428657 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.428659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.428672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.428710 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.429345 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.430074 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.431168 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.431456 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.431493 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.434949 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.434951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.434951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.434958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.436630 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.436964 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.436966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.436966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.437024 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.437283 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.437283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.437284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.437409 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.437561 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.437730 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.437816 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":54,"threads_started":1}
I20260923 22:12:51.438645 7668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:51.440884 7668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:51.443964 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.427923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.1.132.189:45827)
0923 22:12:51.428114 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.428116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.436801 (+ 8685us) server_negotiation.cc:407] Connection header received
0923 22:12:51.436829 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.436830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.436842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.436895 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.437573 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.438645 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.439673 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.439946 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.440022 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.441229 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.441231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.441231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.441238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.442725 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.443096 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.443097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.443097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.443170 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.443359 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.443359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.443360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.443501 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.443619 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.443797 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.443874 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":57,"threads_started":1}
I20260923 22:12:51.448853 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.444809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39467 (local address 127.0.0.1:36932)
0923 22:12:51.445044 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.445052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.445116 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.445532 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.445532 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.445534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.445577 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.445578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.446706 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.446707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.447570 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.447572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.448607 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.448674 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.448678 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.448724 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.448767 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":11,"thread_start_us":81,"threads_started":1}
I20260923 22:12:51.448988 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.444806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33001 (local address 127.0.0.1:60578)
0923 22:12:51.445031 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.445041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.445083 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.446373 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.446373 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.446375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.446409 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.446410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.447388 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.447388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.448079 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.448081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.448759 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.448819 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.448822 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.448875 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.448920 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208}
I20260923 22:12:51.449000 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.444903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.1.132.129:33001)
0923 22:12:51.445773 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.445777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.445786 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.445820 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.445822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.445836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.446368 (+ 532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.446473 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.447328 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448159 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.448683 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448750 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.448812 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.448891 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.448928 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":92,"threads_started":1}
I20260923 22:12:51.449088 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.444929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.1.132.130:39467)
0923 22:12:51.445381 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.445385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.445400 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:12:51.445434 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.445435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.445454 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.445524 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.445657 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.446634 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.447641 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.448417 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448484 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.448559 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.448965 (+ 406us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.449013 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":63,"threads_started":1}
I20260923 22:12:51.449914 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.444925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:41423 (local address 127.0.0.1:50630)
0923 22:12:51.445602 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.445611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.445663 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.445800 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.445800 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.445802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.445836 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.445837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.447651 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.447651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.448704 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.448706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.449601 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.449602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.449729 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.449733 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.449811 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.449849 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":67,"threads_started":1}
I20260923 22:12:51.449945 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.445176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.1.132.131:41423)
0923 22:12:51.445355 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.445361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.445674 (+ 313us) server_negotiation.cc:407] Connection header received
0923 22:12:51.445717 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.445719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.445732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.445796 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.445886 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.446640 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.448787 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.449495 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.449557 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.449625 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.449826 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.449869 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260923 22:12:51.450173 8704 tablet_service.cc:1467] Processing CreateTablet for tablet d6e701c016574bc29252fd59c33ff4c9 (DEFAULT_TABLE table=test-table [id=de4577b19d6c44faa398c05a230fc1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.450178 8567 tablet_service.cc:1467] Processing CreateTablet for tablet d6e701c016574bc29252fd59c33ff4c9 (DEFAULT_TABLE table=test-table [id=de4577b19d6c44faa398c05a230fc1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.450520 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e701c016574bc29252fd59c33ff4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.450524 8704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e701c016574bc29252fd59c33ff4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.452924 8840 tablet_service.cc:1467] Processing CreateTablet for tablet d6e701c016574bc29252fd59c33ff4c9 (DEFAULT_TABLE table=test-table [id=de4577b19d6c44faa398c05a230fc1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:51.453287 8840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e701c016574bc29252fd59c33ff4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:51.453397 8932 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Bootstrap starting.
I20260923 22:12:51.453735 8933 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Bootstrap starting.
I20260923 22:12:51.453997 8932 tablet_bootstrap.cc:654] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.454263 8932 log.cc:824] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:51.454312 8933 tablet_bootstrap.cc:654] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.454597 8933 log.cc:824] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:51.455345 8933 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: No bootstrap required, opened a new log
I20260923 22:12:51.455374 8932 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: No bootstrap required, opened a new log
I20260923 22:12:51.455422 8933 ts_tablet_manager.cc:1397] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:51.455422 8932 ts_tablet_manager.cc:1397] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:51.456987 8936 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Bootstrap starting.
I20260923 22:12:51.457653 8936 tablet_bootstrap.cc:654] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:51.457492 8933 raft_consensus.cc:348] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.457695 8933 raft_consensus.cc:374] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.457727 8933 raft_consensus.cc:729] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae9d80cb9b24908a79c464afc2351f6, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.457876 8933 consensus_queue.cc:260] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.457962 8936 log.cc:824] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:51.457931 8932 raft_consensus.cc:348] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.458077 8932 raft_consensus.cc:374] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.458123 8932 raft_consensus.cc:729] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16eaefabd0f1431bb4beafd5468c84cc, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.458173 8933 ts_tablet_manager.cc:1428] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:51.458247 8932 consensus_queue.cc:260] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.458402 8634 heartbeater.cc:503] Master 127.1.132.190:32829 was elected leader, sending a full tablet report...
I20260923 22:12:51.458504 8932 ts_tablet_manager.cc:1428] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:51.458974 8936 tablet_bootstrap.cc:492] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: No bootstrap required, opened a new log
I20260923 22:12:51.459044 8936 ts_tablet_manager.cc:1397] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:51.459142 8771 heartbeater.cc:503] Master 127.1.132.190:32829 was elected leader, sending a full tablet report...
I20260923 22:12:51.460963 8936 raft_consensus.cc:348] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.461102 8936 raft_consensus.cc:374] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:51.461122 8936 raft_consensus.cc:729] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2820bab0ef3e40b28f8da7fcf4640e20, State: Initialized, Role: FOLLOWER
I20260923 22:12:51.461225 8936 consensus_queue.cc:260] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.461488 8936 ts_tablet_manager.cc:1428] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:12:51.461560 8907 heartbeater.cc:503] Master 127.1.132.190:32829 was elected leader, sending a full tablet report...
W20260923 22:12:51.513806 8773 tablet.cc:2369] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:51.580799 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.578411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.1.132.188:41703)
0923 22:12:51.578464 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.578466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.578475 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.578507 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.578508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.578526 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.578582 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.578712 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.579203 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.580075 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.580505 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.580569 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.580663 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.580685 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.580721 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:51.580801 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.578323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.0.0.1:38724)
0923 22:12:51.578378 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.578389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.578448 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.578592 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.578593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.578594 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.578651 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.578653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.579278 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.579279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.579995 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.579997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.580575 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.580577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.580638 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.580642 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.580682 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.580731 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:12:51.581247 8463 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260923 22:12:51.582119 8941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c970d8fc8d411f81767475089d8ac3. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:51.582232 8941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: LEARNER
W20260923 22:12:51.596701 8636 tablet.cc:2369] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:51.626368 8940 raft_consensus.cc:482] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:51.626503 8940 raft_consensus.cc:504] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.626909 8940 leader_election.cc:288] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467), 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:51.629645 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.627226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:43037 (local address 127.1.132.129:33001)
0923 22:12:51.627265 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.627267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.627276 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:51.627302 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.627302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.627310 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.627357 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.627492 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.628016 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.628961 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.629327 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.629391 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.629400 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.629536 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.629571 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:51.629652 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.627158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:50347 (local address 127.1.132.130:39467)
0923 22:12:51.627214 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.627216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.627240 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:12:51.627295 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.627296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.627306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.627346 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:51.627518 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.627998 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.628956 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.629336 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.629398 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.629406 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.629546 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.629583 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:12:51.629693 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.627099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39467 (local address 127.1.132.131:50347)
0923 22:12:51.627192 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.627204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.627239 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.627349 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.627350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.627352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.627428 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.627430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.628080 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.628081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.628870 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.628872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.629412 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.629413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.629486 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.629489 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.629532 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.629574 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":27}
I20260923 22:12:51.629720 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.627099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33001 (local address 127.1.132.131:43037)
0923 22:12:51.627162 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:51.627175 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:51.627239 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:51.627374 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:51.627375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:51.627376 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:51.627434 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.627435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.628116 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.628117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.628873 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:51.628875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.629400 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.629400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:51.629472 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.629476 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:51.629522 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:12:51.629569 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260923 22:12:51.630280 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e701c016574bc29252fd59c33ff4c9" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16eaefabd0f1431bb4beafd5468c84cc" is_pre_election: true
I20260923 22:12:51.630319 8587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e701c016574bc29252fd59c33ff4c9" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae9d80cb9b24908a79c464afc2351f6" is_pre_election: true
I20260923 22:12:51.630456 8587 raft_consensus.cc:2454] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 0.
I20260923 22:12:51.630457 8724 raft_consensus.cc:2454] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 0.
I20260923 22:12:51.630710 8793 leader_election.cc:302] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2820bab0ef3e40b28f8da7fcf4640e20, 4ae9d80cb9b24908a79c464afc2351f6; no voters:
I20260923 22:12:51.630932 8940 raft_consensus.cc:2781] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:51.630988 8940 raft_consensus.cc:482] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:51.631004 8940 raft_consensus.cc:3037] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.631749 8940 raft_consensus.cc:504] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.631883 8940 leader_election.cc:288] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 election: Requested vote from peers 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467), 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:51.632045 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e701c016574bc29252fd59c33ff4c9" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16eaefabd0f1431bb4beafd5468c84cc"
I20260923 22:12:51.632108 8587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e701c016574bc29252fd59c33ff4c9" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae9d80cb9b24908a79c464afc2351f6"
I20260923 22:12:51.632130 8724 raft_consensus.cc:3037] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.632215 8587 raft_consensus.cc:3037] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:51.632812 8724 raft_consensus.cc:2454] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 1.
I20260923 22:12:51.632968 8587 raft_consensus.cc:2454] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 1.
I20260923 22:12:51.632987 8794 leader_election.cc:302] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16eaefabd0f1431bb4beafd5468c84cc, 2820bab0ef3e40b28f8da7fcf4640e20; no voters:
I20260923 22:12:51.633105 8940 raft_consensus.cc:2781] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:51.633261 8940 raft_consensus.cc:686] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 LEADER]: Becoming Leader. State: Replica: 2820bab0ef3e40b28f8da7fcf4640e20, State: Running, Role: LEADER
I20260923 22:12:51.633345 8940 consensus_queue.cc:237] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:51.634312 7667 catalog_manager.cc:5495] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:51.634807 8416 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: NON_VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: true } } }
I20260923 22:12:51.634898 8416 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.635793 8948 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: attempting to promote NON_VOTER 4ba692dfa53543938b11f1a3f9ab89e0 to VOTER
I20260923 22:12:51.635916 8948 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:51.636426 8276 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 22:12:51.636490 8463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 LEARNER]: Refusing update from remote peer a2c970d8fc8d411f81767475089d8ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 22:12:51.636672 8948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260923 22:12:51.636761 8416 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260923 22:12:51.637316 8418 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.637425 8276 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.638168 8418 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.638197 8950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.638247 8418 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.638286 8950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:51.638252 8948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.638320 8418 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.638362 8418 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.638353 8948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:51.638599 8463 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.639555 8941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c970d8fc8d411f81767475089d8ac3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:51.639645 8941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:51.671741 8909 tablet.cc:2369] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:51.683817 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.681441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.1.132.131:41423)
0923 22:12:51.681476 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.681478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.681484 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:51.681516 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.681516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.681532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.681586 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:51.681801 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.682331 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.683151 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.683339 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.683401 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.683508 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:51.683616 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:51.683674 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.683723 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.683743 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:51.685832 8724 raft_consensus.cc:1261] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Refusing update from remote peer 2820bab0ef3e40b28f8da7fcf4640e20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:51.685880 8587 raft_consensus.cc:1261] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Refusing update from remote peer 2820bab0ef3e40b28f8da7fcf4640e20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:51.686197 8940 consensus_queue.cc:1035] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:51.686358 8946 consensus_queue.cc:1035] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:51.689168 8954 mvcc.cc:205] Tried to move back new op lower bound from 7332665637622312960 to 7332665637410848768. Current Snapshot: MvccSnapshot[applied={T|T < 7332665637622312960}]
I20260923 22:12:51.692586 7667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:51.692687 7667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:51.694077 7667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2c970d8fc8d411f81767475089d8ac3: Sending DeleteTablet for 3 replicas of tablet d6e701c016574bc29252fd59c33ff4c9
I20260923 22:12:51.694434 8840 tablet_service.cc:1514] Processing DeleteTablet for tablet d6e701c016574bc29252fd59c33ff4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:50630
I20260923 22:12:51.694468 8567 tablet_service.cc:1514] Processing DeleteTablet for tablet d6e701c016574bc29252fd59c33ff4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:60578
I20260923 22:12:51.694758 8704 tablet_service.cc:1514] Processing DeleteTablet for tablet d6e701c016574bc29252fd59c33ff4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:51 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36932
I20260923 22:12:51.695093 8964 tablet_replica.cc:330] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: stopping tablet replica
I20260923 22:12:51.695221 8964 raft_consensus.cc:2229] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:51.695341 8964 raft_consensus.cc:2258] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.695708 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 7636
I20260923 22:12:51.696233 8964 ts_tablet_manager.cc:1905] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:51.698061 8964 ts_tablet_manager.cc:1918] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:51.698143 8964 log.cc:1196] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/d6e701c016574bc29252fd59c33ff4c9
I20260923 22:12:51.698411 8964 ts_tablet_manager.cc:1939] T d6e701c016574bc29252fd59c33ff4c9 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting consensus metadata
W20260923 22:12:51.702390 8660 connection.cc:535] server connection from 127.0.0.1:36932 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:12:51.702472 8962 tablet_replica.cc:330] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: stopping tablet replica
I20260923 22:12:51.702596 8962 raft_consensus.cc:2229] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:51.702733 8962 raft_consensus.cc:2258] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.702919 8963 tablet_replica.cc:330] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: stopping tablet replica
I20260923 22:12:51.703013 8963 raft_consensus.cc:2229] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:51.703136 8963 raft_consensus.cc:2258] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:51.703310 8962 ts_tablet_manager.cc:1905] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:51.705165 8962 ts_tablet_manager.cc:1918] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:51.705219 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.696285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.1.132.188:41703)
0923 22:12:51.696327 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.696329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.696475 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:12:51.696581 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.696582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.696592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.696617 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.697378 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.698154 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.699122 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.699328 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.699384 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.700763 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.700765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.700765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.700773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.702045 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.702408 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.702410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.702410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.702467 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.704599 (+ 2132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.704600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.704601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.704745 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.704843 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.705043 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.705133 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:51.705240 8962 log.cc:1196] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/d6e701c016574bc29252fd59c33ff4c9
I20260923 22:12:51.705529 8962 ts_tablet_manager.cc:1939] T d6e701c016574bc29252fd59c33ff4c9 P 4ae9d80cb9b24908a79c464afc2351f6: Deleting consensus metadata
W20260923 22:12:51.705994 8522 connection.cc:441] server connection from 127.0.0.1:60578 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60578 (request call id 1) could send its response
I20260923 22:12:51.706465 8963 ts_tablet_manager.cc:1905] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:51.707967 8963 ts_tablet_manager.cc:1918] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:51.708051 8963 log.cc:1196] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/d6e701c016574bc29252fd59c33ff4c9
I20260923 22:12:51.708338 8963 ts_tablet_manager.cc:1939] T d6e701c016574bc29252fd59c33ff4c9 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting consensus metadata
W20260923 22:12:51.708698 8795 connection.cc:441] server connection from 127.0.0.1:50630 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50630 (request call id 1) could send its response
I20260923 22:12:51.711004 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.704282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.1.132.188:41703)
0923 22:12:51.704330 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.704332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.704340 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:51.704359 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.704359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.704369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.704413 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.705038 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.705816 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.706785 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.706978 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.707046 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.708445 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.708446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.708447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.708454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.709937 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.710316 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.710317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.710317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.710362 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.710556 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.710557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.710557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.710686 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.710779 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.710880 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.710941 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:51.718428 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:51.704484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.1.132.189:45827)
0923 22:12:51.704538 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:51.704540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:51.711136 (+ 6596us) server_negotiation.cc:407] Connection header received
0923 22:12:51.711238 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:51.711239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:51.711256 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:51.711285 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:51.711947 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.712524 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.713607 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:51.713937 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:51.713977 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:51.715646 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:51.715648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:51.715648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:51.715657 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:51.717148 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.717589 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.717591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.717591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.717687 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:51.717908 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:51.717910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:51.717910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:51.718042 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:51.718156 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:51.718278 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:51.718358 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:52.691521 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:52.691248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:48441)
0923 22:12:52.691375 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:52.691459 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
W20260923 22:12:52.691740 8907 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:32829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:52.691761 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:52.691430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:53211)
0923 22:12:52.691586 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:52.691725 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
W20260923 22:12:52.691922 8771 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:32829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:52.692551 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:52.692362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:57915)
0923 22:12:52.692455 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:52.692500 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
W20260923 22:12:52.692730 8634 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:32829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.194619 8975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2c970d8fc8d411f81767475089d8ac3)
I20260923 22:12:53.194782 8975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:53.195025 8975 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703)
I20260923 22:12:53.195278 8976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2c970d8fc8d411f81767475089d8ac3)
I20260923 22:12:53.195374 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37278)
0923 22:12:53.195287 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.195335 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260923 22:12:53.195410 8976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
W20260923 22:12:53.195511 8251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.195677 8976 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189:45827)
I20260923 22:12:53.196031 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37292)
0923 22:12:53.195945 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.195996 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":40,"threads_started":1}
I20260923 22:12:53.196033 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37306)
0923 22:12:53.195981 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.196005 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 22:12:53.196161 8251 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
W20260923 22:12:53.196235 8439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.196630 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.196539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37320)
0923 22:12:53.196588 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.196611 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:12:53.196743 8439 leader_election.cc:334] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.199146 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41703 (local address 127.0.0.1:38742)
0923 22:12:53.195468 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.195478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.195531 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.195902 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.195904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.195906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.195961 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.195964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.196908 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.196908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.198101 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.198104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.198914 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.198974 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.198979 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.199030 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.199076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":45,"threads_started":1}
I20260923 22:12:53.199306 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49542 (local address 127.1.132.189:45827)
0923 22:12:53.195964 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.195965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.196058 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:12:53.196144 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.196145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.196154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.196226 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.196391 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.197017 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198342 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.198897 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198957 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.198966 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.199104 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.199144 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260923 22:12:53.199177 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:45827 (local address 127.0.0.1:49542)
0923 22:12:53.196002 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.196011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:53.196081 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:53.196287 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:53.196288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:53.196290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:53.196334 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.196335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.197084 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.197085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.198259 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:53.198261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.198970 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:53.199026 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.199030 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:53.199076 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:12:53.199118 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":52,"threads_started":1}
I20260923 22:12:53.199376 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.195635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.1.132.188:41703)
0923 22:12:53.195742 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:53.195744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:53.195753 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:53.195779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:53.195780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:53.195790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:53.195851 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:53.196031 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.196820 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198232 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:53.198824 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:53.198898 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:53.198910 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:53.199039 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:12:53.199079 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260923 22:12:53.199648 8276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ecd4c221abd498d845cd89ed3e15f84" is_pre_election: true
I20260923 22:12:53.199744 8276 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ba692dfa53543938b11f1a3f9ab89e0 in term 1.
I20260923 22:12:53.199837 8463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecd4c221abd498d845cd89ed3e15f84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" is_pre_election: true
I20260923 22:12:53.199887 8441 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ba692dfa53543938b11f1a3f9ab89e0, 5ecd4c221abd498d845cd89ed3e15f84; no voters: a2c970d8fc8d411f81767475089d8ac3
I20260923 22:12:53.199954 8463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ecd4c221abd498d845cd89ed3e15f84 in term 1.
I20260923 22:12:53.200021 8976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:12:53.200055 8976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2c970d8fc8d411f81767475089d8ac3)
I20260923 22:12:53.200074 8976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:12:53.200093 8251 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ba692dfa53543938b11f1a3f9ab89e0, 5ecd4c221abd498d845cd89ed3e15f84; no voters: a2c970d8fc8d411f81767475089d8ac3
I20260923 22:12:53.200227 8975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:12:53.200273 8975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2c970d8fc8d411f81767475089d8ac3)
I20260923 22:12:53.200289 8975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:12:53.200778 8976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:53.200883 8976 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 election: Requested vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 5ecd4c221abd498d845cd89ed3e15f84 (127.1.132.189:45827)
I20260923 22:12:53.200978 8975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:53.201038 8276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ecd4c221abd498d845cd89ed3e15f84"
I20260923 22:12:53.201108 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.200999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37346)
0923 22:12:53.201059 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.201084 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:12:53.201123 8975 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 election: Requested vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703)
I20260923 22:12:53.201128 8276 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4ba692dfa53543938b11f1a3f9ab89e0 in current term 2: Already voted for candidate 5ecd4c221abd498d845cd89ed3e15f84 in this term.
I20260923 22:12:53.201345 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.201195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37362)
0923 22:12:53.201297 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.201321 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260923 22:12:53.201373 8463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecd4c221abd498d845cd89ed3e15f84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
I20260923 22:12:53.201411 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.201303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37374)
0923 22:12:53.201362 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.201384 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 22:12:53.201447 8463 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ecd4c221abd498d845cd89ed3e15f84 in current term 2: Already voted for candidate 4ba692dfa53543938b11f1a3f9ab89e0 in this term.
W20260923 22:12:53.201455 8439 leader_election.cc:334] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.201499 8439 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ba692dfa53543938b11f1a3f9ab89e0; no voters: 5ecd4c221abd498d845cd89ed3e15f84, a2c970d8fc8d411f81767475089d8ac3
I20260923 22:12:53.201619 8976 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 22:12:53.201618 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.201540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37378)
0923 22:12:53.201570 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.201595 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260923 22:12:53.201668 8251 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:53.201694 8251 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ecd4c221abd498d845cd89ed3e15f84; no voters: 4ba692dfa53543938b11f1a3f9ab89e0, a2c970d8fc8d411f81767475089d8ac3
I20260923 22:12:53.201771 8975 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 22:12:53.692255 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.692021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:58379)
0923 22:12:53.692136 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.692203 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260923 22:12:53.692395 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.692200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:54117)
0923 22:12:53.692315 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.692349 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260923 22:12:53.693230 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:53.692998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:42475)
0923 22:12:53.693145 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:53.693184 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260923 22:12:54.692829 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.692633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:41297)
0923 22:12:54.692748 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.692785 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260923 22:12:54.692898 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.692721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:36493)
0923 22:12:54.692822 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.692855 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":49,"threads_started":1}
I20260923 22:12:54.693859 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.693656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:55491)
0923 22:12:54.693780 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.693815 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":47,"threads_started":1}
I20260923 22:12:54.949936 8991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:54.950032 8991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:54.950171 8991 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703)
I20260923 22:12:54.950529 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.950348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37608)
0923 22:12:54.950450 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.950493 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":54,"threads_started":1}
I20260923 22:12:54.950593 8463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecd4c221abd498d845cd89ed3e15f84" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" is_pre_election: true
I20260923 22:12:54.950703 8463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ecd4c221abd498d845cd89ed3e15f84 in term 2.
I20260923 22:12:54.950847 8251 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ba692dfa53543938b11f1a3f9ab89e0, 5ecd4c221abd498d845cd89ed3e15f84; no voters:
I20260923 22:12:54.950889 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.950806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37612)
0923 22:12:54.950839 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.950868 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:12:54.950930 8991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 22:12:54.950968 8991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:54.950990 8991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 2 FOLLOWER]: Advancing to term 3
W20260923 22:12:54.951004 8251 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:54.951702 8991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:54.951804 8991 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 election: Requested vote from peers a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829), 4ba692dfa53543938b11f1a3f9ab89e0 (127.1.132.188:41703)
I20260923 22:12:54.952004 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.951907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37622)
0923 22:12:54.951947 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.951980 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:12:54.952068 8463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecd4c221abd498d845cd89ed3e15f84" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4ba692dfa53543938b11f1a3f9ab89e0"
I20260923 22:12:54.952160 8463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:12:54.952328 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.952248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37634)
0923 22:12:54.952284 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.952308 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:12:54.952389 8251 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
I20260923 22:12:54.952728 8463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ecd4c221abd498d845cd89ed3e15f84 in term 3.
I20260923 22:12:54.952875 8251 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ba692dfa53543938b11f1a3f9ab89e0, 5ecd4c221abd498d845cd89ed3e15f84; no voters: a2c970d8fc8d411f81767475089d8ac3
I20260923 22:12:54.952961 8991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:12:54.953071 8991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [term 3 LEADER]: Becoming Leader. State: Replica: 5ecd4c221abd498d845cd89ed3e15f84, State: Running, Role: LEADER
I20260923 22:12:54.953133 8991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } }
I20260923 22:12:54.953470 8994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ecd4c221abd498d845cd89ed3e15f84. Latest consensus state: current_term: 3 leader_uuid: "5ecd4c221abd498d845cd89ed3e15f84" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:54.953557 8994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:54.953711 8996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:54.954069 8996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:54.954118 8996 catalog_manager.cc:1259] Loaded cluster ID: 995fa1ca495d450cb7f5403b5d89b1b6
I20260923 22:12:54.954137 8996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:54.954275 8996 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:54.954324 8996 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Loaded TSK: 0
I20260923 22:12:54.954412 8996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:54.963925 8265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:54.965545 8265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:54.966562 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.966469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37652)
0923 22:12:54.966502 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.966534 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:12:54.966558 8463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [term 3 FOLLOWER]: Refusing update from remote peer 5ecd4c221abd498d845cd89ed3e15f84: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260923 22:12:54.966771 8994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 22:12:54.966912 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.966828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.0.0.1:37668)
0923 22:12:54.966867 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.966891 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:12:54.967000 8251 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 -> Peer a2c970d8fc8d411f81767475089d8ac3 (127.1.132.190:32829): Couldn't send request to peer a2c970d8fc8d411f81767475089d8ac3. Status: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:12:54.967351 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ecd4c221abd498d845cd89ed3e15f84. Latest consensus state: current_term: 3 leader_uuid: "5ecd4c221abd498d845cd89ed3e15f84" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:54.967432 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:54.967701 8997 mvcc.cc:205] Tried to move back new op lower bound from 7332665651061055488 to 7332665651008643072. Current Snapshot: MvccSnapshot[applied={T|T < 7332665651061055488}]
I20260923 22:12:54.967790 8991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5ecd4c221abd498d845cd89ed3e15f84" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:54.967880 8991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:54.967950 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5ecd4c221abd498d845cd89ed3e15f84" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2c970d8fc8d411f81767475089d8ac3" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 32829 } } peers { permanent_uuid: "5ecd4c221abd498d845cd89ed3e15f84" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 45827 } attrs { promote: false } } peers { permanent_uuid: "4ba692dfa53543938b11f1a3f9ab89e0" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41703 } attrs { promote: false } } }
I20260923 22:12:54.968034 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba692dfa53543938b11f1a3f9ab89e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:54.972370 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39467 (local address 127.0.0.1:36946)
0923 22:12:54.969622 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.969632 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.969676 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.970002 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.970003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.970004 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:54.970055 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.970057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.970794 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.970795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.971556 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.971557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.972121 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.972202 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.972206 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.972253 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.972299 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:12:54.972564 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.1.132.130:39467)
0923 22:12:54.969864 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.969867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.969874 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:54.969900 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.969901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.969911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.970000 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:54.970101 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.970723 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.971626 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.971981 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972040 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.972052 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.972451 (+ 399us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.972501 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":42,"threads_started":1}
I20260923 22:12:54.972829 8704 tablet_service.cc:1467] Processing CreateTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 (DEFAULT_TABLE table=test-table [id=ff59bd8c6b4a4c63bb35c848aa3d00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.972959 8704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c97f1c168a4c719b5d0aece7b7a200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.973207 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33001 (local address 127.0.0.1:60580)
0923 22:12:54.969739 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.969748 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.969784 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.970021 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.970022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.970024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:54.970085 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.970087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.970986 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.970986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.972111 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.972113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.972950 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.973033 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.973036 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.973079 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.973119 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260923 22:12:54.973202 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.1.132.129:33001)
0923 22:12:54.969904 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.969906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.969914 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:54.969940 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.969941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.969952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.970020 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:54.970168 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.970900 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972236 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.972838 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972884 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.972895 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.973097 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.973136 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
I20260923 22:12:54.973268 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.1.132.131:41423)
0923 22:12:54.970156 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:54.970158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:54.970165 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:54.970189 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:54.970190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:54.970199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:54.970255 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:54.970399 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.971076 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972288 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.972841 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972907 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.972916 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:54.973157 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 22:12:54.973207 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":43,"threads_started":1}
I20260923 22:12:54.973392 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.969777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:41423 (local address 127.0.0.1:50660)
0923 22:12:54.969877 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.969886 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:54.969921 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:54.970295 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:54.970295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:54.970297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:54.970335 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.970336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.971147 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.971148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.972212 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:54.972214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:54.972938 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:54.972939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:54.973007 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:54.973012 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:54.973064 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:54.973103 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260923 22:12:54.973591 8567 tablet_service.cc:1467] Processing CreateTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 (DEFAULT_TABLE table=test-table [id=ff59bd8c6b4a4c63bb35c848aa3d00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.973655 8840 tablet_service.cc:1467] Processing CreateTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 (DEFAULT_TABLE table=test-table [id=ff59bd8c6b4a4c63bb35c848aa3d00cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:54.973717 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c97f1c168a4c719b5d0aece7b7a200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.973767 8840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c97f1c168a4c719b5d0aece7b7a200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:54.974905 9009 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Bootstrap starting.
I20260923 22:12:54.975172 9011 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Bootstrap starting.
I20260923 22:12:54.975303 9009 tablet_bootstrap.cc:654] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.975534 9011 tablet_bootstrap.cc:654] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.975623 9010 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: Bootstrap starting.
I20260923 22:12:54.975853 9009 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: No bootstrap required, opened a new log
I20260923 22:12:54.975893 9009 ts_tablet_manager.cc:1397] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:54.976006 9011 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: No bootstrap required, opened a new log
I20260923 22:12:54.976054 9011 ts_tablet_manager.cc:1397] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:54.976037 9009 raft_consensus.cc:348] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.976121 9009 raft_consensus.cc:374] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.976147 9009 raft_consensus.cc:729] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16eaefabd0f1431bb4beafd5468c84cc, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.976158 9011 raft_consensus.cc:348] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.976220 9009 consensus_queue.cc:260] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.976248 9011 raft_consensus.cc:374] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.976269 9011 raft_consensus.cc:729] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2820bab0ef3e40b28f8da7fcf4640e20, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.976326 9011 consensus_queue.cc:260] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.976382 9009 ts_tablet_manager.cc:1428] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:54.976480 9011 ts_tablet_manager.cc:1428] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:54.976766 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.976616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:54675)
0923 22:12:54.976706 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.976738 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 22:12:54.976944 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.976767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:48875)
0923 22:12:54.976891 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.976918 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260923 22:12:54.977259 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.977142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:34913)
0923 22:12:54.977176 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.977217 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:12:54.977293 8772 heartbeater.cc:503] Master 127.1.132.189:45827 was elected leader, sending a full tablet report...
I20260923 22:12:54.977626 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.977543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:33377)
0923 22:12:54.977578 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.977603 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:12:54.978071 9010 tablet_bootstrap.cc:654] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:54.978597 9010 tablet_bootstrap.cc:492] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: No bootstrap required, opened a new log
I20260923 22:12:54.978650 9010 ts_tablet_manager.cc:1397] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:12:54.978780 9010 raft_consensus.cc:348] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.978845 9010 raft_consensus.cc:374] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:54.978869 9010 raft_consensus.cc:729] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae9d80cb9b24908a79c464afc2351f6, State: Initialized, Role: FOLLOWER
I20260923 22:12:54.978920 9010 consensus_queue.cc:260] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:54.979089 9010 ts_tablet_manager.cc:1428] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:54.979373 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.979231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:48675)
0923 22:12:54.979320 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.979345 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260923 22:12:54.979494 8635 heartbeater.cc:503] Master 127.1.132.189:45827 was elected leader, sending a full tablet report...
I20260923 22:12:54.979812 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:54.979645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:54535)
0923 22:12:54.979758 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:54.979788 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 22:12:55.169956 9015 raft_consensus.cc:482] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:55.170063 9015 raft_consensus.cc:504] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:55.170208 9015 leader_election.cc:288] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467), 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:55.170463 8587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c97f1c168a4c719b5d0aece7b7a200" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae9d80cb9b24908a79c464afc2351f6" is_pre_election: true
I20260923 22:12:55.170465 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c97f1c168a4c719b5d0aece7b7a200" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16eaefabd0f1431bb4beafd5468c84cc" is_pre_election: true
I20260923 22:12:55.170542 8587 raft_consensus.cc:2454] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 0.
I20260923 22:12:55.170552 8724 raft_consensus.cc:2454] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 0.
I20260923 22:12:55.170697 8793 leader_election.cc:302] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2820bab0ef3e40b28f8da7fcf4640e20, 4ae9d80cb9b24908a79c464afc2351f6; no voters:
I20260923 22:12:55.170800 9015 raft_consensus.cc:2781] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:55.170840 9015 raft_consensus.cc:482] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:55.170853 9015 raft_consensus.cc:3037] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:55.171494 9015 raft_consensus.cc:504] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:55.171635 9015 leader_election.cc:288] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 election: Requested vote from peers 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467), 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001)
I20260923 22:12:55.171825 8587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c97f1c168a4c719b5d0aece7b7a200" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae9d80cb9b24908a79c464afc2351f6"
I20260923 22:12:55.171850 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c97f1c168a4c719b5d0aece7b7a200" candidate_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16eaefabd0f1431bb4beafd5468c84cc"
I20260923 22:12:55.171893 8587 raft_consensus.cc:3037] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:55.171911 8724 raft_consensus.cc:3037] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:55.172547 8724 raft_consensus.cc:2454] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 1.
I20260923 22:12:55.172612 8587 raft_consensus.cc:2454] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2820bab0ef3e40b28f8da7fcf4640e20 in term 1.
I20260923 22:12:55.172729 8794 leader_election.cc:302] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16eaefabd0f1431bb4beafd5468c84cc, 2820bab0ef3e40b28f8da7fcf4640e20; no voters:
I20260923 22:12:55.172868 9015 raft_consensus.cc:2781] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:55.172993 9015 raft_consensus.cc:686] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 LEADER]: Becoming Leader. State: Replica: 2820bab0ef3e40b28f8da7fcf4640e20, State: Running, Role: LEADER
I20260923 22:12:55.173038 9015 consensus_queue.cc:237] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } }
I20260923 22:12:55.173132 8908 heartbeater.cc:503] Master 127.1.132.189:45827 was elected leader, sending a full tablet report...
I20260923 22:12:55.173319 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.173221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:35843)
0923 22:12:55.173256 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.173287 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 22:12:55.173622 8265 catalog_manager.cc:5495] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2820bab0ef3e40b28f8da7fcf4640e20" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41423 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:55.199904 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.197741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.1.132.131:41423)
0923 22:12:55.197805 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:55.197807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:55.197815 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:55.197839 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:55.197840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:55.197849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:55.197900 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:55.198022 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:55.198511 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:55.199243 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:55.199441 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:55.199496 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:55.199669 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:55.199703 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:55.199745 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:55.199798 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:12:55.199823 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:12:55.200881 8587 raft_consensus.cc:1261] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Refusing update from remote peer 2820bab0ef3e40b28f8da7fcf4640e20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:55.200886 8724 raft_consensus.cc:1261] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Refusing update from remote peer 2820bab0ef3e40b28f8da7fcf4640e20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:55.201107 9015 consensus_queue.cc:1035] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16eaefabd0f1431bb4beafd5468c84cc" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:55.201233 9018 consensus_queue.cc:1035] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae9d80cb9b24908a79c464afc2351f6" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:55.201728 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.201621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:46809)
0923 22:12:55.201663 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.201695 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:12:55.201984 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.201884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:52895)
0923 22:12:55.201924 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.201957 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:12:55.203152 9024 mvcc.cc:205] Tried to move back new op lower bound from 7332665652020858880 to 7332665651909320704. Current Snapshot: MvccSnapshot[applied={T|T < 7332665652020858880}]
I20260923 22:12:55.203573 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.203345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.129:47283)
0923 22:12:55.203505 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.203534 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260923 22:12:55.203572 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.203431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:35811)
0923 22:12:55.203496 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.203538 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:12:55.203578 9021 mvcc.cc:205] Tried to move back new op lower bound from 7332665652020858880 to 7332665651909320704. Current Snapshot: MvccSnapshot[applied={T|T < 7332665652020858880}]
I20260923 22:12:55.204278 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.204149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.130:45297)
0923 22:12:55.204214 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.204247 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:12:55.204768 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:55.204684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:32829 (local address 127.1.132.131:39805)
0923 22:12:55.204714 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.204738 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:32829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:12:55.206267 8266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:55.206370 8266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:55.208834 8266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ecd4c221abd498d845cd89ed3e15f84: Sending DeleteTablet for 3 replicas of tablet b3c97f1c168a4c719b5d0aece7b7a200
I20260923 22:12:55.209254 8567 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:55 UTC) from {username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:60580
I20260923 22:12:55.209414 8704 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:55 UTC) from {username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:36946
I20260923 22:12:55.209571 9031 tablet_replica.cc:330] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: stopping tablet replica
I20260923 22:12:55.209626 9031 raft_consensus.cc:2229] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:55.209619 8840 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c97f1c168a4c719b5d0aece7b7a200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:55 UTC) from {username='oryx', principal='oryx/127.1.132.189@KRBTEST.COM'} at 127.0.0.1:50660
I20260923 22:12:55.209664 9031 raft_consensus.cc:2258] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:55.209772 9032 tablet_replica.cc:330] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: stopping tablet replica
I20260923 22:12:55.209853 9032 raft_consensus.cc:2229] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:55.209914 9032 raft_consensus.cc:2258] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:55.210022 9031 ts_tablet_manager.cc:1905] T b3c97f1c168a4c719b5d0aece7b7a200 P 4ae9d80cb9b24908a79c464afc2351f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:55.210253 9032 ts_tablet_manager.cc:1905] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:55.211105 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 8413
I20260923 22:12:55.212301 9033 tablet_replica.cc:330] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: stopping tablet replica
I20260923 22:12:55.212365 9033 raft_consensus.cc:2229] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:55.212440 9033 raft_consensus.cc:2258] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:55.212603 9032 ts_tablet_manager.cc:1918] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:55.212656 9032 log.cc:1196] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/b3c97f1c168a4c719b5d0aece7b7a200
I20260923 22:12:55.212862 9032 ts_tablet_manager.cc:1939] T b3c97f1c168a4c719b5d0aece7b7a200 P 16eaefabd0f1431bb4beafd5468c84cc: Deleting consensus metadata
I20260923 22:12:55.213755 9033 ts_tablet_manager.cc:1905] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:55.213979 8251 catalog_manager.cc:4841] TS 16eaefabd0f1431bb4beafd5468c84cc (127.1.132.130:39467): tablet b3c97f1c168a4c719b5d0aece7b7a200 (table test-table [id=ff59bd8c6b4a4c63bb35c848aa3d00cc]) successfully deleted
I20260923 22:12:55.215238 9033 ts_tablet_manager.cc:1918] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:55.215281 9033 log.cc:1196] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/b3c97f1c168a4c719b5d0aece7b7a200
I20260923 22:12:55.215481 9033 ts_tablet_manager.cc:1939] T b3c97f1c168a4c719b5d0aece7b7a200 P 2820bab0ef3e40b28f8da7fcf4640e20: Deleting consensus metadata
I20260923 22:12:55.215828 8250 catalog_manager.cc:4841] TS 2820bab0ef3e40b28f8da7fcf4640e20 (127.1.132.131:41423): tablet b3c97f1c168a4c719b5d0aece7b7a200 (table test-table [id=ff59bd8c6b4a4c63bb35c848aa3d00cc]) successfully deleted
W20260923 22:12:55.217391 9006 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:55.217255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33001 (local address 127.0.0.1:60584)
0923 22:12:55.217309 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:55.217338 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:33001: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:55.217543 8250 catalog_manager.cc:4584] TS 4ae9d80cb9b24908a79c464afc2351f6 (127.1.132.129:33001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3c97f1c168a4c719b5d0aece7b7a200: Network error: Client connection negotiation failed: client connection to 127.1.132.129:33001: connect: Connection reset by peer (error 104)
I20260923 22:12:55.217698 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 8641
I20260923 22:12:55.222131 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 8777
I20260923 22:12:55.227813 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 8132
I20260923 22:12:55.231946 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 8320
2026-09-23T22:12:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6547 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:55 dist-test-slave-l4q5 krb5kdc[9036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:55 dist-test-slave-l4q5 krb5kdc[9036](info): set up 2 sockets
Sep 23 22:12:55 dist-test-slave-l4q5 krb5kdc[9036](info): commencing operation
krb5kdc: starting...
W20260923 22:12:56.196635 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab.
Entry for principal oryx/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab.
I20260923 22:12:56.315330 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:44377
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab
--principal=oryx/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:44377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:56.396023 9049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:56.396301 9049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:56.396337 9049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:56.397549 9049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:56.397703 9049 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:12:56.397770 9049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:56.397794 9049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:56.397818 9049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:56.397836 9049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:56.399627 9049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:44377
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.190.keytab
--principal=oryx/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:44377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:56.399923 9049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:56.400151 9049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:56.404848 9054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.404872 9057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.404875 9055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.404963 9049 server_base.cc:1034] running on GCE node
I20260923 22:12:56.405313 9049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:56.405581 9049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:56.406769 9049 hybrid_clock.cc:648] HybridClock initialized: now 1790201576406771 us; error 46 us; skew 500 ppm
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:56.409179 9049 init.cc:377] Logged in from keytab as oryx/127.1.132.190@KRBTEST.COM (short username oryx)
I20260923 22:12:56.409610 9049 webserver.cc:460] Webserver started at http://127.1.132.190:33725/ using document root <none> and password file <none>
I20260923 22:12:56.409813 9049 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:56.409861 9049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:56.409982 9049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:56.411008 9049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "6ced7e076bf34c7ab96f254ece3c6640"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "3ca36db1f7380e559ba04ccd85d010a5"
server_key_iv: "444ccbc071295c66bda6b361e74a1a69"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.411330 9049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "6ced7e076bf34c7ab96f254ece3c6640"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "3ca36db1f7380e559ba04ccd85d010a5"
server_key_iv: "444ccbc071295c66bda6b361e74a1a69"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.412582 9049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:12:56.413264 9064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.413415 9049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:56.413494 9049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "6ced7e076bf34c7ab96f254ece3c6640"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "3ca36db1f7380e559ba04ccd85d010a5"
server_key_iv: "444ccbc071295c66bda6b361e74a1a69"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.413553 9049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:56.429929 9049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:56.431758 9049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:56.431895 9049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:56.436232 9049 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:44377
I20260923 22:12:56.436322 9126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:44377 every 8 connection(s)
I20260923 22:12:56.436605 9049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:56.437218 9127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:56.439383 9127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Bootstrap starting.
I20260923 22:12:56.439973 9127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:56.440052 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9049
I20260923 22:12:56.440143 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:12:56.440301 9127 log.cc:824] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:56.440351 1554 external_mini_cluster.cc:1428] Setting key 1689479bdd12247fb18a66e7affa3a8f
I20260923 22:12:56.440995 9127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: No bootstrap required, opened a new log
I20260923 22:12:56.443084 9127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } }
I20260923 22:12:56.443230 9127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:56.443261 9127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ced7e076bf34c7ab96f254ece3c6640, State: Initialized, Role: FOLLOWER
I20260923 22:12:56.443387 9127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } }
I20260923 22:12:56.443459 9127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:56.443488 9127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:56.443535 9127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:56.444077 9127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } }
I20260923 22:12:56.444199 9127 leader_election.cc:302] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ced7e076bf34c7ab96f254ece3c6640; no voters:
I20260923 22:12:56.444350 9127 leader_election.cc:288] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:56.444396 9132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:56.444574 9127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:56.444577 9132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [term 1 LEADER]: Becoming Leader. State: Replica: 6ced7e076bf34c7ab96f254ece3c6640, State: Running, Role: LEADER
I20260923 22:12:56.444983 9132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } }
I20260923 22:12:56.446386 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ced7e076bf34c7ab96f254ece3c6640. Latest consensus state: current_term: 1 leader_uuid: "6ced7e076bf34c7ab96f254ece3c6640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } } }
I20260923 22:12:56.446589 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ced7e076bf34c7ab96f254ece3c6640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ced7e076bf34c7ab96f254ece3c6640" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44377 } } }
I20260923 22:12:56.446683 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:56.446714 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640 [sys.catalog]: This master's current role is: LEADER
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
W20260923 22:12:56.446939 9146 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:56.446991 9146 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:56.447331 9148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:56.447489 9148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:56.449195 9148 catalog_manager.cc:1347] Generated new cluster ID: 2a4b9bb21d0d49bcb0681a2d51dbba01
I20260923 22:12:56.449242 9148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:56.450652 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.440820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.1.132.190:44377)
0923 22:12:56.440990 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.440999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.441027 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:12:56.441464 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.441470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.441518 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.441646 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.442241 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.443113 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.443984 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.444568 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.444674 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.447290 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.447294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.447294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.447304 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.449037 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.449442 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.449446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.449446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.449552 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.449748 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.449748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.449749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.450063 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.450206 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.450309 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.450453 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260923 22:12:56.466084 9148 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:56.466631 9148 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:56.472033 9148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Generated new TSK 0
I20260923 22:12:56.472254 9148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
I20260923 22:12:56.504563 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44377
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:56.586344 9153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:56.586555 9153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:56.586577 9153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:56.587739 9153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:56.587782 9153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:56.587801 9153 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:56.587817 9153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:12:56.589668 9153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:56.589999 9153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:56.590274 9153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:56.595038 9161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.595142 9153 server_base.cc:1034] running on GCE node
W20260923 22:12:56.595038 9159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.595059 9158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.595458 9153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:56.595713 9153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:56.596861 9153 hybrid_clock.cc:648] HybridClock initialized: now 1790201576596847 us; error 25 us; skew 500 ppm
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:56.599221 9153 init.cc:377] Logged in from keytab as oryx/127.1.132.129@KRBTEST.COM (short username oryx)
I20260923 22:12:56.599666 9153 webserver.cc:460] Webserver started at http://127.1.132.129:44783/ using document root <none> and password file <none>
I20260923 22:12:56.599882 9153 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:56.599929 9153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:56.600036 9153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:56.601136 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "378bf167f63c4002ad2d2abe7079c37a"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "c88f45463762e927ec7e738acd750341"
server_key_iv: "0da1e13fd8e977d24860c87a9dab303a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.601486 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "378bf167f63c4002ad2d2abe7079c37a"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "c88f45463762e927ec7e738acd750341"
server_key_iv: "0da1e13fd8e977d24860c87a9dab303a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.602684 9153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:56.603323 9168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.603470 9153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:56.603546 9153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "378bf167f63c4002ad2d2abe7079c37a"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "c88f45463762e927ec7e738acd750341"
server_key_iv: "0da1e13fd8e977d24860c87a9dab303a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.603649 9153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:56.613147 9153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:56.615103 9153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:56.615223 9153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:56.615790 9153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:56.615837 9153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.615867 9153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:56.615896 9153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:56.622972 9153 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:32815
I20260923 22:12:56.623041 9283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:32815 every 8 connection(s)
I20260923 22:12:56.623381 9153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:12:56.626210 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.616213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:42943 (local address 127.1.132.190:44377)
0923 22:12:56.616274 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.616277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.616606 (+ 329us) server_negotiation.cc:407] Connection header received
0923 22:12:56.617245 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.617246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.617260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.617341 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.618256 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.618913 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.620498 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.620719 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.620778 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.623404 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.623406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.623406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.623415 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.624928 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.625308 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.625310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.625310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.625365 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.625573 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.625574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.625575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.625690 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.625773 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.626033 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.626121 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:56.626389 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.616226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.129:42943)
0923 22:12:56.616386 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.616440 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.617138 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.617427 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.617433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.617811 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.618129 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.618137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.619073 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.619074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.620405 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.620408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.620871 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.620872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.621029 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.621656 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.621666 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.623305 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.625009 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.625016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.625018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.625240 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.625417 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.625418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.625418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.625521 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.625775 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.625781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.625915 (+ 134us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.626005 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.626168 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260923 22:12:56.626840 9284 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44377
I20260923 22:12:56.626920 9284 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:56.627091 9284 heartbeater.cc:511] Master 127.1.132.190:44377 requested a full tablet report, sending...
I20260923 22:12:56.627600 9080 ts_manager.cc:194] Registered new tserver with Master: 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815)
I20260923 22:12:56.628443 9080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:42943
I20260923 22:12:56.629200 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9153
I20260923 22:12:56.629314 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:12:56.629504 1554 external_mini_cluster.cc:1428] Setting key e2a56f6c1d48c30dc65459a0e75f296b
I20260923 22:12:56.633445 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.627874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:49667 (local address 127.1.132.190:44377)
0923 22:12:56.627924 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.627926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.627970 (+ 44us) server_negotiation.cc:407] Connection header received
0923 22:12:56.628031 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.628032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.628041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.628092 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.628395 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.629020 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.629831 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.630130 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.630200 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.631083 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.631084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.631084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.631091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.632324 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.632691 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.632692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.632692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.632741 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.632939 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.632940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.632940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.633055 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.633120 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.633285 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.633349 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:56.633466 9290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.627800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.129:49667)
0923 22:12:56.627917 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.627927 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.627981 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.628095 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.628096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.628289 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.628338 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.628340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.629107 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.629108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.629758 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.629761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.630223 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.630224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.630296 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.630707 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.630712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.631001 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.632394 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.632395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.632396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.632584 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.632795 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.632796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.632797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.632857 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.633119 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.633120 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.633169 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.633265 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.633368 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
I20260923 22:12:56.655896 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44377
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:56.735018 9293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:56.735252 9293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:56.735288 9293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:56.736536 9293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:56.736583 9293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:56.736611 9293 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:56.736634 9293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:12:56.738200 9293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:56.738435 9293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:56.738649 9293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:56.743640 9298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.743695 9301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.743784 9293 server_base.cc:1034] running on GCE node
W20260923 22:12:56.743655 9299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.744127 9293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:56.744402 9293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:56.745535 9293 hybrid_clock.cc:648] HybridClock initialized: now 1790201576745525 us; error 30 us; skew 500 ppm
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:56.747859 9293 init.cc:377] Logged in from keytab as oryx/127.1.132.130@KRBTEST.COM (short username oryx)
I20260923 22:12:56.748330 9293 webserver.cc:460] Webserver started at http://127.1.132.130:45715/ using document root <none> and password file <none>
I20260923 22:12:56.748509 9293 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:56.748548 9293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:56.748639 9293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:56.749658 9293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "c12eaa35664c443eac0d2e552c415572"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "efc5b8894fb3ea56fcce446093ebc038"
server_key_iv: "37b756c09a82ec620c2ce7a8aeefdbe1"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.749948 9293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "c12eaa35664c443eac0d2e552c415572"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "efc5b8894fb3ea56fcce446093ebc038"
server_key_iv: "37b756c09a82ec620c2ce7a8aeefdbe1"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.751230 9293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:56.751896 9308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.752053 9293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.752132 9293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "c12eaa35664c443eac0d2e552c415572"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "efc5b8894fb3ea56fcce446093ebc038"
server_key_iv: "37b756c09a82ec620c2ce7a8aeefdbe1"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.752230 9293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:56.771641 9293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:56.773816 9293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:56.773934 9293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:56.774443 9293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:56.774487 9293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.774516 9293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:56.774534 9293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:56.781343 9293 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:36757
I20260923 22:12:56.781432 9423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:36757 every 8 connection(s)
I20260923 22:12:56.781735 9293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:12:56.783794 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.774867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:47353 (local address 127.1.132.190:44377)
0923 22:12:56.774922 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.774924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.775123 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:12:56.775641 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.775642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.775657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.775732 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.776490 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.777255 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.778434 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.778661 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.778723 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.781151 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.781153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.781154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.781163 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.782585 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.782943 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.782944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.782944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.782990 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.783199 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.783200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.783200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.783328 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.783392 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.783647 (+ 255us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.783724 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:56.783902 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.774850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.130:47353)
0923 22:12:56.775004 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.775042 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.775530 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.775813 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.775817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.776106 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.776396 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.776401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.777349 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.777350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.778284 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.778287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.778753 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.778753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.778884 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.779422 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.779433 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.781018 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.782665 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.782677 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.782678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.782886 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.783034 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.783034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.783034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.783120 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.783386 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.783389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.783503 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.783582 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.783734 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260923 22:12:56.784390 9424 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44377
I20260923 22:12:56.784469 9424 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:56.784644 9424 heartbeater.cc:511] Master 127.1.132.190:44377 requested a full tablet report, sending...
I20260923 22:12:56.785008 9080 ts_manager.cc:194] Registered new tserver with Master: c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757)
I20260923 22:12:56.785373 9080 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:47353
I20260923 22:12:56.790310 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.785397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:39525 (local address 127.1.132.190:44377)
0923 22:12:56.785441 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.785443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.785500 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:12:56.785575 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.785576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.785585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.785626 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.785921 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.786385 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.787067 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.787260 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.787310 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.788121 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.788122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.788122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.788128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.789256 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.789551 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.789552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.789553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.789642 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.789805 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.789806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.789806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.789942 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.789997 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.790150 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.790218 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:56.790323 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.785334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.130:39525)
0923 22:12:56.785442 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.785451 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.785523 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.785629 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.785630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.785800 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.785855 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.785857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.786465 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.786466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.786988 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.786990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.787313 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.787314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.787378 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.787762 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.787765 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.788022 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.789337 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.789338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.789339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.789492 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.789698 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.789698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.789699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.789755 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.789997 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.789998 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.790043 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.790118 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.790220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":59,"threads_started":1}
I20260923 22:12:56.790756 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9293
I20260923 22:12:56.790861 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:12:56.791078 1554 external_mini_cluster.cc:1428] Setting key c5ef92a36599c07cd6e46e4ab9c1ea12
WARNING: no policy specified for oryx/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
I20260923 22:12:56.815836 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44377
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:56.896270 9433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:56.896471 9433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:56.896493 9433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:56.897624 9433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:56.897665 9433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:56.897683 9433 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:12:56.897698 9433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:12:56.899196 9433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.132.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:56.899412 9433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:56.899628 9433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:56.904539 9441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.904539 9439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:56.904553 9438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:56.904757 9433 server_base.cc:1034] running on GCE node
I20260923 22:12:56.904999 9433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:56.905239 9433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:56.906384 9433 hybrid_clock.cc:648] HybridClock initialized: now 1790201576906369 us; error 29 us; skew 500 ppm
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:56.908775 9433 init.cc:377] Logged in from keytab as oryx/127.1.132.131@KRBTEST.COM (short username oryx)
I20260923 22:12:56.909195 9433 webserver.cc:460] Webserver started at http://127.1.132.131:43481/ using document root <none> and password file <none>
I20260923 22:12:56.909368 9433 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:56.909404 9433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:56.909493 9433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:56.910514 9433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "b30cea8fd9b500c165a782aca38200a4"
server_key_iv: "537597262920c8bd0c33203cf02a1e7a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.910802 9433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "b30cea8fd9b500c165a782aca38200a4"
server_key_iv: "537597262920c8bd0c33203cf02a1e7a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.912032 9433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:56.912887 9448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.913084 9433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:56.913195 9433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
format_stamp: "Formatted at 2026-09-23 22:12:56 on dist-test-slave-l4q5"
server_key: "b30cea8fd9b500c165a782aca38200a4"
server_key_iv: "537597262920c8bd0c33203cf02a1e7a"
server_key_version: "encryptionkey@0"
I20260923 22:12:56.913278 9433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:56.931773 9433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:56.933889 9433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:56.934028 9433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:56.934547 9433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:12:56.934587 9433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:56.934617 9433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:12:56.934639 9433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:56.941578 9433 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:43657
I20260923 22:12:56.941653 9563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:43657 every 8 connection(s)
I20260923 22:12:56.942116 9433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:12:56.944068 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.934992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:38065 (local address 127.1.132.190:44377)
0923 22:12:56.935079 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.935081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.935270 (+ 189us) server_negotiation.cc:407] Connection header received
0923 22:12:56.935937 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.935938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.935948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.935990 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.936887 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.937401 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.938475 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.938665 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.938721 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.941273 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.941275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.941276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.941285 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.942788 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.943170 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.943171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.943171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.943223 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.943432 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.943433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.943433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.943545 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.943610 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.943866 (+ 256us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.943978 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 22:12:56.944248 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.934967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.131:38065)
0923 22:12:56.935126 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.935179 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.935839 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.936071 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.936076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.936472 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.936787 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.936795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.937496 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.937497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.938386 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.938389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.938754 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.938755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.938877 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.939382 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.939392 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.941151 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.942869 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.942875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.942877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.943101 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.943277 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.943278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.943278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.943379 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.943606 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.943612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.943747 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.943840 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.944006 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260923 22:12:56.944695 9564 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44377
I20260923 22:12:56.944787 9564 heartbeater.cc:464] Registering TS with master...
I20260923 22:12:56.944964 9564 heartbeater.cc:511] Master 127.1.132.190:44377 requested a full tablet report, sending...
I20260923 22:12:56.945361 9081 ts_manager.cc:194] Registered new tserver with Master: d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:56.945870 9081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:38065
I20260923 22:12:56.950692 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9433
I20260923 22:12:56.950826 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:12:56.951012 1554 external_mini_cluster.cc:1428] Setting key 9926c0a5f39f2aeb4f8da88689a82a8e
I20260923 22:12:56.951239 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.945751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:48161 (local address 127.1.132.190:44377)
0923 22:12:56.945794 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.945796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.945900 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:12:56.945941 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.945941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.945950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.946009 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.946731 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.947272 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.948003 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.948229 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.948296 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.949109 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.949110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.949110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.949116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.950185 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.950469 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.950470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.950470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.950537 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.950744 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.950745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.950745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.950869 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.950925 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.951088 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.951153 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:12:56.951251 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.945720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.1.132.131:48161)
0923 22:12:56.945834 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.945844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.945898 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.946017 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.946018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.946617 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:56.946665 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.946668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.947387 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.947389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.947914 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.947916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.948296 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.948297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.948349 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.948762 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:56.948766 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:56.949029 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:56.950259 (+ 1230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.950260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.950261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.950410 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.950622 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:56.950623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:56.950623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:56.950689 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:56.950926 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:56.950927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:56.950975 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.951032 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.951137 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260923 22:12:56.951916 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:12:56.973517 9131 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:56.967925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.1.132.190:44377)
0923 22:12:56.967977 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.967978 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.967987 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:56.968012 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.968013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.968022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.968109 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.968813 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.969338 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.970131 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.970314 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.970365 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.973446 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45352: BlockingRecv error: recv got EOF from 127.0.0.1:45352 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 22:12:56 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:56.981125 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.974352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.1.132.190:44377)
0923 22:12:56.974407 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.974409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.974452 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:12:56.974526 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.974527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.974536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.974580 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:56.975059 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.975535 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.976221 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.976405 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.976458 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.978543 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:56.978545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:56.978545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:56.978555 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:56.979996 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.980413 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.980414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.980415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.980468 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:56.980644 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:56.980645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:56.980645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:56.980757 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:56.980842 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.980947 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.981028 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:12:56.983165 9080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:56.985028 9080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:56.992455 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.988071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.1.132.131:43657)
0923 22:12:56.988382 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.988385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.988395 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:56.988446 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.988448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.988464 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.988535 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:56.988798 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.990024 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.991345 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.991882 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.991950 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992031 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.992307 (+ 276us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.992355 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260923 22:12:56.992432 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.987831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43657 (local address 127.0.0.1:36976)
0923 22:12:56.988256 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.988295 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.988351 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.988624 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.988627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.988630 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:56.988718 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.988721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.990137 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.990138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.991268 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.991271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.991964 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.991965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.992059 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992074 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.992145 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.992336 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260923 22:12:56.993125 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.987831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:36757 (local address 127.0.0.1:51796)
0923 22:12:56.988281 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.988311 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.988378 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.988625 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.988630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.988632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:56.988729 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.988731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.989747 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.989748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.991049 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.991052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.992865 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.992865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.992920 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992923 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.992983 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.993033 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":26,"thread_start_us":71,"threads_started":1}
I20260923 22:12:56.993139 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.988081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.1.132.129:32815)
0923 22:12:56.988308 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.988312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.988527 (+ 215us) server_negotiation.cc:407] Connection header received
0923 22:12:56.988686 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.988689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.988705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.988775 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:56.988961 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.990160 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.992008 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.992767 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.992824 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992881 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.993004 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.993041 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260923 22:12:56.993220 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.988032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.1.132.130:36757)
0923 22:12:56.988233 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:56.988238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:56.988386 (+ 148us) server_negotiation.cc:407] Connection header received
0923 22:12:56.988449 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:56.988451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:56.988474 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:56.988544 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:56.988808 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.989656 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.991140 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.992263 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.992329 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992396 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:56.993077 (+ 681us) server_negotiation.cc:299] Negotiation successful
0923 22:12:56.993121 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260923 22:12:56.993125 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:56.987830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:32815 (local address 127.0.0.1:34244)
0923 22:12:56.988451 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:56.988459 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:56.988510 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:56.988836 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:56.988836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:56.988839 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:56.988888 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.988889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.990235 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.990236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.991896 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:56.991899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:56.992841 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:56.992843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:56.992920 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:56.992926 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:12:56.992989 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:12:56.993033 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":11,"thread_start_us":97,"threads_started":1}
I20260923 22:12:56.994040 9498 tablet_service.cc:1467] Processing CreateTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 (DEFAULT_TABLE table=test-table [id=47f41e75d737432aa123bf41ba4ea5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:56.994069 9358 tablet_service.cc:1467] Processing CreateTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 (DEFAULT_TABLE table=test-table [id=47f41e75d737432aa123bf41ba4ea5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:56.994233 9218 tablet_service.cc:1467] Processing CreateTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 (DEFAULT_TABLE table=test-table [id=47f41e75d737432aa123bf41ba4ea5f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:56.994405 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfa7960b2f949b8b6a938ac9003b907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:56.994414 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfa7960b2f949b8b6a938ac9003b907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:56.994526 9218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfa7960b2f949b8b6a938ac9003b907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:56.997460 9592 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Bootstrap starting.
I20260923 22:12:56.998066 9592 tablet_bootstrap.cc:654] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:56.998358 9592 log.cc:824] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:56.998531 9593 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Bootstrap starting.
I20260923 22:12:56.999073 9593 tablet_bootstrap.cc:654] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:56.999078 9592 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: No bootstrap required, opened a new log
I20260923 22:12:56.999131 9592 ts_tablet_manager.cc:1397] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:56.999336 9593 log.cc:824] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:56.999670 9594 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Bootstrap starting.
I20260923 22:12:57.000027 9593 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: No bootstrap required, opened a new log
I20260923 22:12:57.000131 9593 ts_tablet_manager.cc:1397] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:57.000408 9594 tablet_bootstrap.cc:654] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.000723 9594 log.cc:824] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:57.001237 9592 raft_consensus.cc:348] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.001381 9592 raft_consensus.cc:374] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.001411 9592 raft_consensus.cc:729] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.001502 9594 tablet_bootstrap.cc:492] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: No bootstrap required, opened a new log
I20260923 22:12:57.001574 9594 ts_tablet_manager.cc:1397] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:12:57.001565 9592 consensus_queue.cc:260] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.001854 9592 ts_tablet_manager.cc:1428] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:57.001981 9564 heartbeater.cc:503] Master 127.1.132.190:44377 was elected leader, sending a full tablet report...
I20260923 22:12:57.002079 9593 raft_consensus.cc:348] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.002249 9593 raft_consensus.cc:374] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.002327 9593 raft_consensus.cc:729] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12eaa35664c443eac0d2e552c415572, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.002470 9593 consensus_queue.cc:260] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.002720 9593 ts_tablet_manager.cc:1428] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 22:12:57.002835 9424 heartbeater.cc:503] Master 127.1.132.190:44377 was elected leader, sending a full tablet report...
I20260923 22:12:57.003856 9594 raft_consensus.cc:348] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.003981 9594 raft_consensus.cc:374] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.004001 9594 raft_consensus.cc:729] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.004114 9594 consensus_queue.cc:260] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.004377 9594 ts_tablet_manager.cc:1428] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:12:57.004427 9284 heartbeater.cc:503] Master 127.1.132.190:44377 was elected leader, sending a full tablet report...
W20260923 22:12:57.032316 9425 tablet.cc:2369] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:57.112857 9600 raft_consensus.cc:482] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:57.112983 9600 raft_consensus.cc:504] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.113351 9600 leader_election.cc:288] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.117194 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.113672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58019 (local address 127.1.132.131:43657)
0923 22:12:57.113719 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.113721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.113790 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:12:57.113820 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.113821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.113830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.113880 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:57.114000 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.114734 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116230 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.116840 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116903 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.116913 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.117067 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.117122 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:57.117198 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.113656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:40055 (local address 127.1.132.130:36757)
0923 22:12:57.113702 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.113704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.113717 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:12:57.113749 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.113750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.113759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.113823 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:57.113967 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.114722 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116225 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.116806 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116903 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.116913 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.117067 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.117120 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:57.117274 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.113544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43657 (local address 127.1.132.129:58019)
0923 22:12:57.113725 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.113733 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.113784 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.113891 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.113892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.113893 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:57.113944 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.113945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.114835 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.114837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.116105 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.116110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.116903 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.116994 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.116999 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.117057 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.117098 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":121,"thread_start_us":72,"threads_started":1}
I20260923 22:12:57.117277 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.113543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:36757 (local address 127.1.132.129:40055)
0923 22:12:57.113589 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.113640 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.113692 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.113833 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.113834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.113836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:57.113905 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.113907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.114836 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.114837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.116106 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.116110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.116903 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.116904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.116996 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.116999 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.117057 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.117098 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":12}
I20260923 22:12:57.117686 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfa7960b2f949b8b6a938ac9003b907" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" is_pre_election: true
I20260923 22:12:57.117786 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfa7960b2f949b8b6a938ac9003b907" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572" is_pre_election: true
I20260923 22:12:57.117839 9518 raft_consensus.cc:2454] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.117926 9378 raft_consensus.cc:2454] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.118052 9172 leader_election.cc:302] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.118201 9600 raft_consensus.cc:2781] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:57.118258 9600 raft_consensus.cc:482] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:57.118281 9600 raft_consensus.cc:3037] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.119166 9600 raft_consensus.cc:504] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.119288 9600 leader_election.cc:288] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Requested vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.119442 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfa7960b2f949b8b6a938ac9003b907" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572"
I20260923 22:12:57.119525 9378 raft_consensus.cc:3037] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.119522 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfa7960b2f949b8b6a938ac9003b907" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
I20260923 22:12:57.119597 9518 raft_consensus.cc:3037] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.120285 9518 raft_consensus.cc:2454] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.120288 9378 raft_consensus.cc:2454] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.120466 9169 leader_election.cc:302] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, c12eaa35664c443eac0d2e552c415572; no voters:
I20260923 22:12:57.120577 9600 raft_consensus.cc:2781] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.120724 9600 raft_consensus.cc:686] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Becoming Leader. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Running, Role: LEADER
I20260923 22:12:57.120810 9600 consensus_queue.cc:237] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } }
I20260923 22:12:57.121797 9080 catalog_manager.cc:5495] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a reported cstate change: term changed from 0 to 1, leader changed from <none> to 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129). New cstate: current_term: 1 leader_uuid: "378bf167f63c4002ad2d2abe7079c37a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } health_report { overall_health: HEALTHY } } }
W20260923 22:12:57.124125 9285 tablet.cc:2369] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:57.137120 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.134920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.1.132.129:32815)
0923 22:12:57.134975 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.134977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.134985 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:57.135024 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.135025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.135045 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.135104 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.135267 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.135775 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.136533 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.136728 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.136785 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.136876 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.136973 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.137025 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.137038 (+ 13us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.137061 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:12:57.138747 9378 raft_consensus.cc:1261] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.138808 9518 raft_consensus.cc:1261] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.139132 9600 consensus_queue.cc:1035] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.139263 9602 consensus_queue.cc:1035] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.141410 9605 mvcc.cc:205] Tried to move back new op lower bound from 7332665659957870592 to 7332665659887820800. Current Snapshot: MvccSnapshot[applied={T|T < 7332665659957870592}]
I20260923 22:12:57.141601 9607 mvcc.cc:205] Tried to move back new op lower bound from 7332665659957870592 to 7332665659887820800. Current Snapshot: MvccSnapshot[applied={T|T < 7332665659957870592}]
I20260923 22:12:57.141716 9606 mvcc.cc:205] Tried to move back new op lower bound from 7332665659957870592 to 7332665659887820800. Current Snapshot: MvccSnapshot[applied={T|T < 7332665659957870592}]
I20260923 22:12:57.143797 9080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.143877 9080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.144522 9080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Sending DeleteTablet for 3 replicas of tablet bcfa7960b2f949b8b6a938ac9003b907
I20260923 22:12:57.144829 9498 tablet_service.cc:1514] Processing DeleteTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36976
I20260923 22:12:57.144857 9218 tablet_service.cc:1514] Processing DeleteTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34244
I20260923 22:12:57.144930 9358 tablet_service.cc:1514] Processing DeleteTablet for tablet bcfa7960b2f949b8b6a938ac9003b907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51796
I20260923 22:12:57.145195 9617 tablet_replica.cc:330] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: stopping tablet replica
I20260923 22:12:57.145223 9616 tablet_replica.cc:330] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: stopping tablet replica
I20260923 22:12:57.145334 9617 raft_consensus.cc:2229] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.145280 9080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 22:12:57.145358 9616 raft_consensus.cc:2229] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 22:12:57.145407 9080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:57.145704 9617 raft_consensus.cc:2258] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.145740 9616 raft_consensus.cc:2258] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.146183 9617 ts_tablet_manager.cc:1905] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.146412 9616 ts_tablet_manager.cc:1905] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.146724 9358 tablet_service.cc:1467] Processing CreateTablet for tablet 94f05e8d070f45559e08fdba39f34c68 (DEFAULT_TABLE table=test-table [id=60dc332c2973442183657f31c3da9990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.146828 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f05e8d070f45559e08fdba39f34c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.147890 9615 tablet_replica.cc:330] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: stopping tablet replica
I20260923 22:12:57.147986 9615 raft_consensus.cc:2229] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:57.148078 9593 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Bootstrap starting.
I20260923 22:12:57.148079 9498 tablet_service.cc:1467] Processing CreateTablet for tablet 94f05e8d070f45559e08fdba39f34c68 (DEFAULT_TABLE table=test-table [id=60dc332c2973442183657f31c3da9990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.148113 9615 raft_consensus.cc:2258] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.148514 9593 tablet_bootstrap.cc:654] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.148587 9615 ts_tablet_manager.cc:1905] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.148625 9218 tablet_service.cc:1467] Processing CreateTablet for tablet 94f05e8d070f45559e08fdba39f34c68 (DEFAULT_TABLE table=test-table [id=60dc332c2973442183657f31c3da9990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.148708 9218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f05e8d070f45559e08fdba39f34c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.148926 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94f05e8d070f45559e08fdba39f34c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.149935 9615 ts_tablet_manager.cc:1918] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.149997 9615 log.cc:1196] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/bcfa7960b2f949b8b6a938ac9003b907
I20260923 22:12:57.150089 9592 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Bootstrap starting.
I20260923 22:12:57.150098 9594 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Bootstrap starting.
I20260923 22:12:57.150235 9615 ts_tablet_manager.cc:1939] T bcfa7960b2f949b8b6a938ac9003b907 P 378bf167f63c4002ad2d2abe7079c37a: Deleting consensus metadata
I20260923 22:12:57.150460 9592 tablet_bootstrap.cc:654] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.150484 9594 tablet_bootstrap.cc:654] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.150818 9067 catalog_manager.cc:4841] TS 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815): tablet bcfa7960b2f949b8b6a938ac9003b907 (table test-table [id=47f41e75d737432aa123bf41ba4ea5f7]) successfully deleted
I20260923 22:12:57.150978 9592 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: No bootstrap required, opened a new log
I20260923 22:12:57.151022 9592 ts_tablet_manager.cc:1397] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.151039 9616 ts_tablet_manager.cc:1918] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.151104 9616 log.cc:1196] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/bcfa7960b2f949b8b6a938ac9003b907
I20260923 22:12:57.151152 9592 raft_consensus.cc:348] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.151201 9592 raft_consensus.cc:374] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.151212 9592 raft_consensus.cc:729] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.151245 9592 consensus_queue.cc:260] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.151319 9592 ts_tablet_manager.cc:1428] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.151335 9616 ts_tablet_manager.cc:1939] T bcfa7960b2f949b8b6a938ac9003b907 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting consensus metadata
I20260923 22:12:57.151656 9068 catalog_manager.cc:4841] TS d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657): tablet bcfa7960b2f949b8b6a938ac9003b907 (table test-table [id=47f41e75d737432aa123bf41ba4ea5f7]) successfully deleted
I20260923 22:12:57.151696 9617 ts_tablet_manager.cc:1918] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.151760 9617 log.cc:1196] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/bcfa7960b2f949b8b6a938ac9003b907
I20260923 22:12:57.152069 9617 ts_tablet_manager.cc:1939] T bcfa7960b2f949b8b6a938ac9003b907 P c12eaa35664c443eac0d2e552c415572: Deleting consensus metadata
I20260923 22:12:57.152580 9594 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: No bootstrap required, opened a new log
I20260923 22:12:57.152632 9594 ts_tablet_manager.cc:1397] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:12:57.152706 9593 tablet_bootstrap.cc:492] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: No bootstrap required, opened a new log
I20260923 22:12:57.152753 9593 ts_tablet_manager.cc:1397] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:12:57.152760 9594 raft_consensus.cc:348] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.152828 9594 raft_consensus.cc:374] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.152853 9594 raft_consensus.cc:729] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.152896 9594 consensus_queue.cc:260] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.152894 9593 raft_consensus.cc:348] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.152956 9593 raft_consensus.cc:374] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.152979 9593 raft_consensus.cc:729] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12eaa35664c443eac0d2e552c415572, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.152997 9594 ts_tablet_manager.cc:1428] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.153016 9593 consensus_queue.cc:260] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.153092 9593 ts_tablet_manager.cc:1428] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:57.153080 9065 catalog_manager.cc:4841] TS c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757): tablet bcfa7960b2f949b8b6a938ac9003b907 (table test-table [id=47f41e75d737432aa123bf41ba4ea5f7]) successfully deleted
W20260923 22:12:57.192708 9565 tablet.cc:2369] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:57.364612 9610 raft_consensus.cc:482] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:57.364722 9610 raft_consensus.cc:504] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.364888 9610 leader_election.cc:288] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.365125 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f05e8d070f45559e08fdba39f34c68" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" is_pre_election: true
I20260923 22:12:57.365130 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f05e8d070f45559e08fdba39f34c68" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572" is_pre_election: true
I20260923 22:12:57.365222 9518 raft_consensus.cc:2454] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.365221 9378 raft_consensus.cc:2454] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.365370 9172 leader_election.cc:302] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.365459 9610 raft_consensus.cc:2781] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:57.365520 9610 raft_consensus.cc:482] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:57.365541 9610 raft_consensus.cc:3037] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.366251 9610 raft_consensus.cc:504] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.366374 9610 leader_election.cc:288] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Requested vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.366529 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f05e8d070f45559e08fdba39f34c68" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572"
I20260923 22:12:57.366556 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f05e8d070f45559e08fdba39f34c68" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
I20260923 22:12:57.366607 9378 raft_consensus.cc:3037] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.366621 9518 raft_consensus.cc:3037] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.367326 9518 raft_consensus.cc:2454] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.367326 9378 raft_consensus.cc:2454] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.367542 9172 leader_election.cc:302] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.367620 9610 raft_consensus.cc:2781] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.367666 9610 raft_consensus.cc:686] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Becoming Leader. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Running, Role: LEADER
I20260923 22:12:57.367709 9610 consensus_queue.cc:237] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.368299 9081 catalog_manager.cc:5495] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a reported cstate change: term changed from 0 to 1, leader changed from <none> to 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129). New cstate: current_term: 1 leader_uuid: "378bf167f63c4002ad2d2abe7079c37a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.190@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:57.411697 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.405080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.1.132.190:44377)
0923 22:12:57.405137 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.405140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.405276 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:12:57.405346 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.405347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.405356 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.405395 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.405870 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.406367 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.407062 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.407250 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.407308 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.409319 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.409321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.409321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.409327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.410575 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.410930 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.410930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.410931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.410979 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.411138 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.411140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.411140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.411245 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.411310 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.411436 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.411519 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:12:57.411763 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.405006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44377 (local address 127.0.0.1:45382)
0923 22:12:57.405206 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.405216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.405286 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.405409 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.405410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.405738 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:57.405798 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.405801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.406434 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.406435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.406996 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.406998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.407307 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.407308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.407355 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.407829 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:57.407833 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:57.409232 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:57.410668 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.410677 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.410680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.410864 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.411031 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:57.411031 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:57.411031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:57.411090 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:57.411299 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:57.411307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:57.411352 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.411410 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.411574 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":61,"thread_start_us":100,"threads_started":1}
I20260923 22:12:57.414104 9081 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:45382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:12:57.414297 9081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:57.415552 9218 tablet_service.cc:1467] Processing CreateTablet for tablet 820e6d465ca241afae2785c8ad3cbd01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4eebef4765482598b6046f5a66e3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:57.415583 9358 tablet_service.cc:1467] Processing CreateTablet for tablet 820e6d465ca241afae2785c8ad3cbd01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4eebef4765482598b6046f5a66e3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:57.415637 9498 tablet_service.cc:1467] Processing CreateTablet for tablet 820e6d465ca241afae2785c8ad3cbd01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d4eebef4765482598b6046f5a66e3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:12:57.415726 9218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820e6d465ca241afae2785c8ad3cbd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.415730 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820e6d465ca241afae2785c8ad3cbd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.415745 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820e6d465ca241afae2785c8ad3cbd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.417073 9594 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: Bootstrap starting.
I20260923 22:12:57.417248 9593 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: Bootstrap starting.
I20260923 22:12:57.417408 9594 tablet_bootstrap.cc:654] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.417605 9593 tablet_bootstrap.cc:654] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.417881 9592 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: Bootstrap starting.
I20260923 22:12:57.418001 9594 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: No bootstrap required, opened a new log
I20260923 22:12:57.418045 9594 ts_tablet_manager.cc:1397] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.418176 9593 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: No bootstrap required, opened a new log
I20260923 22:12:57.418164 9594 raft_consensus.cc:348] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.418215 9593 ts_tablet_manager.cc:1397] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:57.418228 9592 tablet_bootstrap.cc:654] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.418236 9594 raft_consensus.cc:374] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.418257 9594 raft_consensus.cc:729] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.418311 9594 consensus_queue.cc:260] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.418357 9593 raft_consensus.cc:348] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.418398 9594 ts_tablet_manager.cc:1428] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.418414 9593 raft_consensus.cc:374] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.418432 9593 raft_consensus.cc:729] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12eaa35664c443eac0d2e552c415572, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.418426 9610 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.418471 9593 consensus_queue.cc:260] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.418511 9610 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.418547 9593 ts_tablet_manager.cc:1428] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.418651 9599 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.418735 9599 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.419242 9592 tablet_bootstrap.cc:492] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: No bootstrap required, opened a new log
I20260923 22:12:57.419292 9592 ts_tablet_manager.cc:1397] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.419416 9592 raft_consensus.cc:348] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.419467 9592 raft_consensus.cc:374] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.419479 9592 raft_consensus.cc:729] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.419508 9592 consensus_queue.cc:260] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.419610 9592 ts_tablet_manager.cc:1428] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.419682 9598 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.419752 9598 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.550110 9598 raft_consensus.cc:482] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:57.550220 9598 raft_consensus.cc:504] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.550566 9598 leader_election.cc:288] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815), c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757)
I20260923 22:12:57.553424 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.550906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:58395 (local address 127.1.132.130:36757)
0923 22:12:57.550969 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.550971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.550983 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:57.551066 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.551067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.551076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.551127 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:57.551266 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.551739 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.552778 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.553128 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.553189 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.553198 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.553320 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.553355 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:12:57.553445 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.550766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:36757 (local address 127.1.132.131:58395)
0923 22:12:57.550922 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.550935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.550996 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.551134 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.551135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.551136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:57.551211 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.551213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.551816 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.551817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.552695 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.552698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.553201 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.553202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.553263 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.553267 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.553310 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.553350 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":18,"thread_start_us":72,"threads_started":1}
I20260923 22:12:57.553503 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.550846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:51681 (local address 127.1.132.129:32815)
0923 22:12:57.550897 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.550899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.551066 (+ 167us) server_negotiation.cc:407] Connection header received
0923 22:12:57.551117 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.551118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.551129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.551187 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:12:57.551315 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.551771 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.552787 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.553204 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.553264 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.553271 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.553403 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.553440 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:57.553532 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.550770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:32815 (local address 127.1.132.131:51681)
0923 22:12:57.550998 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.551007 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.551064 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.551191 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.551192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.551193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:12:57.551257 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.551259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.551842 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.551843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.552707 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.552709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.553269 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.553270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.553345 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.553348 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.553392 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.553432 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":18,"thread_start_us":64,"threads_started":1}
I20260923 22:12:57.553802 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820e6d465ca241afae2785c8ad3cbd01" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572" is_pre_election: true
I20260923 22:12:57.553890 9378 raft_consensus.cc:2454] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 0.
I20260923 22:12:57.554000 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820e6d465ca241afae2785c8ad3cbd01" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378bf167f63c4002ad2d2abe7079c37a" is_pre_election: true
I20260923 22:12:57.554073 9449 leader_election.cc:302] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c12eaa35664c443eac0d2e552c415572, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.554116 9238 raft_consensus.cc:2454] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 0.
I20260923 22:12:57.554196 9598 raft_consensus.cc:2781] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:57.554235 9598 raft_consensus.cc:482] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:57.554251 9598 raft_consensus.cc:3037] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.554944 9598 raft_consensus.cc:504] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.555063 9598 leader_election.cc:288] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 election: Requested vote from peers 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815), c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757)
I20260923 22:12:57.555262 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820e6d465ca241afae2785c8ad3cbd01" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378bf167f63c4002ad2d2abe7079c37a"
I20260923 22:12:57.555336 9238 raft_consensus.cc:3037] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.555330 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820e6d465ca241afae2785c8ad3cbd01" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572"
I20260923 22:12:57.555404 9378 raft_consensus.cc:3037] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.555962 9378 raft_consensus.cc:2454] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 1.
I20260923 22:12:57.556109 9449 leader_election.cc:302] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c12eaa35664c443eac0d2e552c415572, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.556135 9238 raft_consensus.cc:2454] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 1.
I20260923 22:12:57.556221 9598 raft_consensus.cc:2781] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.556348 9598 raft_consensus.cc:686] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 LEADER]: Becoming Leader. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Running, Role: LEADER
I20260923 22:12:57.556419 9598 consensus_queue.cc:237] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.556710 9633 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: TxnStatusTablet state changed. Reason: New leader d5c87f133eb448c2b49a5b7a6c311f21. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.556789 9633 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:57.557031 9635 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:12:57.557114 9635 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:12:57.557087 9081 catalog_manager.cc:5495] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:57.567374 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.565039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36980 (local address 127.1.132.131:43657)
0923 22:12:57.565084 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.565086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.565094 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:57.565181 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.565183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.565201 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.565254 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.565363 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.565877 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.566619 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.566820 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.566874 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.567038 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.567150 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.567210 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.567279 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.567299 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:57.567391 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.564971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43657 (local address 127.0.0.1:36980)
0923 22:12:57.565047 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.565060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.565107 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.565258 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.565259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.565260 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:57.565310 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.565312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.565958 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.565960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.566544 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.566547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.566885 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.566886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.566951 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.566963 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.567209 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.567216 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.567273 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.567324 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:12:57.568356 9378 raft_consensus.cc:1261] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Refusing update from remote peer d5c87f133eb448c2b49a5b7a6c311f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.568408 9238 raft_consensus.cc:1261] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Refusing update from remote peer d5c87f133eb448c2b49a5b7a6c311f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.568609 9633 consensus_queue.cc:1035] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.568756 9598 consensus_queue.cc:1035] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.569180 9599 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: TxnStatusTablet state changed. Reason: New leader d5c87f133eb448c2b49a5b7a6c311f21. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.569271 9599 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.569355 9610 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: TxnStatusTablet state changed. Reason: New leader d5c87f133eb448c2b49a5b7a6c311f21. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.569428 9610 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.569859 9633 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.569931 9633 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:57.570266 9599 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.570341 9599 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P c12eaa35664c443eac0d2e552c415572: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.571292 9598 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.571372 9598 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P d5c87f133eb448c2b49a5b7a6c311f21: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:12:57.571720 9610 tablet_replica.cc:439] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } } }
I20260923 22:12:57.571799 9610 tablet_replica.cc:442] T 820e6d465ca241afae2785c8ad3cbd01 P 378bf167f63c4002ad2d2abe7079c37a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:12:57.576273 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.573524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58109 (local address 127.1.132.131:43657)
0923 22:12:57.573624 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.573626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.573717 (+ 91us) server_negotiation.cc:407] Connection header received
0923 22:12:57.573785 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.573785 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.573793 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.573834 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.573941 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.574503 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.575652 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.575836 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.575903 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.576060 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.576084 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.576118 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.576166 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.576205 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 22:12:57.576319 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.573495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43657 (local address 127.1.132.129:58109)
0923 22:12:57.573666 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.573677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.573731 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.573837 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.573838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.573840 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:57.573891 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.573892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.574581 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.574582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.575560 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.575564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.575910 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.575911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.575971 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.575987 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.576120 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.576122 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.576155 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.576237 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260923 22:12:57.578187 9518 raft_consensus.cc:1261] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.578208 9378 raft_consensus.cc:1261] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.578361 9602 consensus_queue.cc:1035] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.578459 9610 consensus_queue.cc:1035] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.586812 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.583327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:32815 (local address 127.1.132.131:48261)
0923 22:12:57.583466 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.583476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:57.583513 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:57.583659 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:57.583660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:57.583661 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:12:57.583742 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.583743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.584666 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.584667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.586020 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:57.586023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.586470 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.586470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:57.586525 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.586539 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.586687 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.586688 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:12:57.586709 (+ 21us) client_negotiation.cc:240] Negotiation successful
0923 22:12:57.586740 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260923 22:12:57.586807 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.583494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:48261 (local address 127.1.132.129:32815)
0923 22:12:57.583529 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.583531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.583538 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:57.583569 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.583570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.583580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.583639 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.583822 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.584592 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.586092 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.586390 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.586459 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.586604 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.586638 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.586694 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.586713 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.586732 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:57.594982 9081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.595105 9081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.595769 9081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Sending DeleteTablet for 3 replicas of tablet 94f05e8d070f45559e08fdba39f34c68
I20260923 22:12:57.596051 9218 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f05e8d070f45559e08fdba39f34c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34244
I20260923 22:12:57.596037 9358 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f05e8d070f45559e08fdba39f34c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51796
I20260923 22:12:57.596153 9615 tablet_replica.cc:330] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: stopping tablet replica
I20260923 22:12:57.596174 9617 tablet_replica.cc:330] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: stopping tablet replica
I20260923 22:12:57.596247 9617 raft_consensus.cc:2229] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.596247 9615 raft_consensus.cc:2229] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:57.596285 9617 raft_consensus.cc:2258] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.596307 9615 raft_consensus.cc:2258] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.596650 9617 ts_tablet_manager.cc:1905] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.596706 9615 ts_tablet_manager.cc:1905] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.598228 9498 tablet_service.cc:1514] Processing DeleteTablet for tablet 94f05e8d070f45559e08fdba39f34c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36976
I20260923 22:12:57.598335 9616 tablet_replica.cc:330] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: stopping tablet replica
I20260923 22:12:57.598363 9615 ts_tablet_manager.cc:1918] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:57.598381 9616 raft_consensus.cc:2229] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.598416 9615 log.cc:1196] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/94f05e8d070f45559e08fdba39f34c68
I20260923 22:12:57.598418 9616 raft_consensus.cc:2258] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.598546 9617 ts_tablet_manager.cc:1918] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:57.598591 9617 log.cc:1196] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/94f05e8d070f45559e08fdba39f34c68
I20260923 22:12:57.598615 9615 ts_tablet_manager.cc:1939] T 94f05e8d070f45559e08fdba39f34c68 P 378bf167f63c4002ad2d2abe7079c37a: Deleting consensus metadata
I20260923 22:12:57.598774 9616 ts_tablet_manager.cc:1905] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.598800 9617 ts_tablet_manager.cc:1939] T 94f05e8d070f45559e08fdba39f34c68 P c12eaa35664c443eac0d2e552c415572: Deleting consensus metadata
I20260923 22:12:57.599143 9067 catalog_manager.cc:4841] TS 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815): tablet 94f05e8d070f45559e08fdba39f34c68 (table test-table [id=60dc332c2973442183657f31c3da9990]) successfully deleted
I20260923 22:12:57.599125 9065 catalog_manager.cc:4841] TS c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757): tablet 94f05e8d070f45559e08fdba39f34c68 (table test-table [id=60dc332c2973442183657f31c3da9990]) successfully deleted
I20260923 22:12:57.600425 9616 ts_tablet_manager.cc:1918] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:57.600488 9616 log.cc:1196] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/94f05e8d070f45559e08fdba39f34c68
I20260923 22:12:57.600692 9616 ts_tablet_manager.cc:1939] T 94f05e8d070f45559e08fdba39f34c68 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting consensus metadata
I20260923 22:12:57.601212 9068 catalog_manager.cc:4841] TS d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657): tablet 94f05e8d070f45559e08fdba39f34c68 (table test-table [id=60dc332c2973442183657f31c3da9990]) successfully deleted
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:12:57.621053 9131 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:57.614736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.1.132.190:44377)
0923 22:12:57.614794 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.614796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.614910 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:12:57.614944 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.614945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.614956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.615013 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.615749 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.616427 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.617712 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.617899 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.617956 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.620980 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45386: BlockingRecv error: recv got EOF from 127.0.0.1:45386 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM
I20260923 22:12:57.628615 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.621936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.1.132.190:44377)
0923 22:12:57.621978 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.621980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.622059 (+ 79us) server_negotiation.cc:407] Connection header received
0923 22:12:57.622122 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.622123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.622132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.622181 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:57.622706 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.623223 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.623897 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.624082 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.624137 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.626221 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:57.626223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:57.626223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:57.626232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:57.627489 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.627816 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.627817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.627818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.627867 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:57.628070 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:57.628071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:57.628071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:57.628227 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:57.628320 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.628444 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.628524 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:57.630390 9081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:57.630576 9081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:57.631891 9218 tablet_service.cc:1467] Processing CreateTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (DEFAULT_TABLE table=test-table [id=0f4b3cc2cd7947308d15f5c97d165397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.631947 9498 tablet_service.cc:1467] Processing CreateTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (DEFAULT_TABLE table=test-table [id=0f4b3cc2cd7947308d15f5c97d165397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.631980 9358 tablet_service.cc:1467] Processing CreateTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (DEFAULT_TABLE table=test-table [id=0f4b3cc2cd7947308d15f5c97d165397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.632042 9218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.632078 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.632081 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.633384 9593 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Bootstrap starting.
I20260923 22:12:57.633730 9592 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Bootstrap starting.
I20260923 22:12:57.633796 9593 tablet_bootstrap.cc:654] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.634006 9592 tablet_bootstrap.cc:654] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.634148 9594 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Bootstrap starting.
I20260923 22:12:57.634398 9594 tablet_bootstrap.cc:654] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.634482 9593 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: No bootstrap required, opened a new log
I20260923 22:12:57.634521 9593 ts_tablet_manager.cc:1397] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:57.634668 9593 raft_consensus.cc:348] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.634742 9593 raft_consensus.cc:374] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.634763 9593 raft_consensus.cc:729] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12eaa35664c443eac0d2e552c415572, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.634801 9593 consensus_queue.cc:260] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.634891 9593 ts_tablet_manager.cc:1428] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.635393 9594 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: No bootstrap required, opened a new log
I20260923 22:12:57.635443 9594 ts_tablet_manager.cc:1397] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.635540 9594 raft_consensus.cc:348] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.635587 9594 raft_consensus.cc:374] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.635598 9594 raft_consensus.cc:729] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.635632 9594 consensus_queue.cc:260] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.635699 9594 ts_tablet_manager.cc:1428] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.636314 9592 tablet_bootstrap.cc:492] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: No bootstrap required, opened a new log
I20260923 22:12:57.636360 9592 ts_tablet_manager.cc:1397] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260923 22:12:57.636468 9592 raft_consensus.cc:348] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.636523 9592 raft_consensus.cc:374] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.636539 9592 raft_consensus.cc:729] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.636579 9592 consensus_queue.cc:260] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.636664 9592 ts_tablet_manager.cc:1428] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.699684 9640 raft_consensus.cc:482] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:57.699796 9640 raft_consensus.cc:504] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.700039 9640 leader_election.cc:288] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815), c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757)
I20260923 22:12:57.700309 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558fdd8d451c4cfe9c89e2c25c36c6ef" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378bf167f63c4002ad2d2abe7079c37a" is_pre_election: true
I20260923 22:12:57.700317 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558fdd8d451c4cfe9c89e2c25c36c6ef" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572" is_pre_election: true
I20260923 22:12:57.700407 9378 raft_consensus.cc:2454] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 0.
I20260923 22:12:57.700413 9238 raft_consensus.cc:2454] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 0.
I20260923 22:12:57.700613 9451 leader_election.cc:302] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.700697 9640 raft_consensus.cc:2781] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:57.700737 9640 raft_consensus.cc:482] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:57.700757 9640 raft_consensus.cc:3037] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.701545 9640 raft_consensus.cc:504] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.701682 9640 leader_election.cc:288] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 election: Requested vote from peers 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815), c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757)
I20260923 22:12:57.701968 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558fdd8d451c4cfe9c89e2c25c36c6ef" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572"
I20260923 22:12:57.701984 9238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558fdd8d451c4cfe9c89e2c25c36c6ef" candidate_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378bf167f63c4002ad2d2abe7079c37a"
I20260923 22:12:57.702050 9378 raft_consensus.cc:3037] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.702060 9238 raft_consensus.cc:3037] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.702678 9378 raft_consensus.cc:2454] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 1.
I20260923 22:12:57.702780 9238 raft_consensus.cc:2454] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c87f133eb448c2b49a5b7a6c311f21 in term 1.
I20260923 22:12:57.702844 9449 leader_election.cc:302] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c12eaa35664c443eac0d2e552c415572, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.702921 9640 raft_consensus.cc:2781] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.702953 9640 raft_consensus.cc:686] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 LEADER]: Becoming Leader. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Running, Role: LEADER
I20260923 22:12:57.703001 9640 consensus_queue.cc:237] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.703606 9081 catalog_manager.cc:5495] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } health_report { overall_health: HEALTHY } } }
I20260923 22:12:57.713236 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.710779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.1.132.131:43657)
0923 22:12:57.710828 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.710830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.710840 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:57.710868 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.710869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.710880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.710935 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.711122 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.711597 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.712479 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.712689 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.712755 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.712943 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.712974 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.713032 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.713107 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.713127 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:12:57.714082 9378 raft_consensus.cc:1261] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Refusing update from remote peer d5c87f133eb448c2b49a5b7a6c311f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.714092 9238 raft_consensus.cc:1261] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Refusing update from remote peer d5c87f133eb448c2b49a5b7a6c311f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.714308 9633 consensus_queue.cc:1035] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.714421 9640 consensus_queue.cc:1035] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.718110 9079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.718220 9079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.718891 9079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Sending DeleteTablet for 3 replicas of tablet 558fdd8d451c4cfe9c89e2c25c36c6ef
I20260923 22:12:57.719198 9218 tablet_service.cc:1514] Processing DeleteTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34244
I20260923 22:12:57.719200 9358 tablet_service.cc:1514] Processing DeleteTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51796
I20260923 22:12:57.719291 9615 tablet_replica.cc:330] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: stopping tablet replica
I20260923 22:12:57.719292 9617 tablet_replica.cc:330] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: stopping tablet replica
I20260923 22:12:57.719336 9617 raft_consensus.cc:2229] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.719340 9615 raft_consensus.cc:2229] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.719380 9615 raft_consensus.cc:2258] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.719389 9617 raft_consensus.cc:2258] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.719403 9079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:12:57.719563 9079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:12:57.719756 9617 ts_tablet_manager.cc:1905] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.719826 9498 tablet_service.cc:1514] Processing DeleteTablet for tablet 558fdd8d451c4cfe9c89e2c25c36c6ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36976
I20260923 22:12:57.719908 9616 tablet_replica.cc:330] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: stopping tablet replica
I20260923 22:12:57.719947 9616 raft_consensus.cc:2229] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:57.720041 9616 raft_consensus.cc:2258] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.720068 9615 ts_tablet_manager.cc:1905] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.721369 9616 ts_tablet_manager.cc:1905] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.721485 9615 ts_tablet_manager.cc:1918] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.722010 9617 ts_tablet_manager.cc:1918] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.722076 9617 log.cc:1196] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/558fdd8d451c4cfe9c89e2c25c36c6ef
I20260923 22:12:57.722256 9617 ts_tablet_manager.cc:1939] T 558fdd8d451c4cfe9c89e2c25c36c6ef P c12eaa35664c443eac0d2e552c415572: Deleting consensus metadata
I20260923 22:12:57.721561 9615 log.cc:1196] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/558fdd8d451c4cfe9c89e2c25c36c6ef
I20260923 22:12:57.722558 9065 catalog_manager.cc:4841] TS c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757): tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (table test-table [id=0f4b3cc2cd7947308d15f5c97d165397]) successfully deleted
I20260923 22:12:57.722643 9615 ts_tablet_manager.cc:1939] T 558fdd8d451c4cfe9c89e2c25c36c6ef P 378bf167f63c4002ad2d2abe7079c37a: Deleting consensus metadata
I20260923 22:12:57.722888 9616 ts_tablet_manager.cc:1918] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:12:57.722947 9616 log.cc:1196] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/558fdd8d451c4cfe9c89e2c25c36c6ef
I20260923 22:12:57.722930 9067 catalog_manager.cc:4841] TS 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815): tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (table test-table [id=0f4b3cc2cd7947308d15f5c97d165397]) successfully deleted
I20260923 22:12:57.723131 9616 ts_tablet_manager.cc:1939] T 558fdd8d451c4cfe9c89e2c25c36c6ef P d5c87f133eb448c2b49a5b7a6c311f21: Deleting consensus metadata
I20260923 22:12:57.723659 9218 tablet_service.cc:1467] Processing CreateTablet for tablet 42fd00f95ec74673861f884373791098 (DEFAULT_TABLE table=test-table [id=df7dbb01aecc48bbad52328799f93056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.723659 9358 tablet_service.cc:1467] Processing CreateTablet for tablet 42fd00f95ec74673861f884373791098 (DEFAULT_TABLE table=test-table [id=df7dbb01aecc48bbad52328799f93056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.723762 9218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42fd00f95ec74673861f884373791098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.723762 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42fd00f95ec74673861f884373791098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.724933 9593 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Bootstrap starting.
I20260923 22:12:57.725063 9068 catalog_manager.cc:4841] TS d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657): tablet 558fdd8d451c4cfe9c89e2c25c36c6ef (table test-table [id=0f4b3cc2cd7947308d15f5c97d165397]) successfully deleted
I20260923 22:12:57.725224 9498 tablet_service.cc:1467] Processing CreateTablet for tablet 42fd00f95ec74673861f884373791098 (DEFAULT_TABLE table=test-table [id=df7dbb01aecc48bbad52328799f93056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:12:57.725320 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42fd00f95ec74673861f884373791098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:57.725606 9593 tablet_bootstrap.cc:654] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.725917 9594 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: Bootstrap starting.
I20260923 22:12:57.726184 9594 tablet_bootstrap.cc:654] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.726264 9592 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Bootstrap starting.
I20260923 22:12:57.726528 9592 tablet_bootstrap.cc:654] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:57.726624 9593 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: No bootstrap required, opened a new log
I20260923 22:12:57.726672 9593 ts_tablet_manager.cc:1397] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260923 22:12:57.726816 9593 raft_consensus.cc:348] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.726888 9593 raft_consensus.cc:374] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.726908 9593 raft_consensus.cc:729] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12eaa35664c443eac0d2e552c415572, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.726963 9593 consensus_queue.cc:260] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.727006 9592 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: No bootstrap required, opened a new log
I20260923 22:12:57.727044 9592 ts_tablet_manager.cc:1397] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:12:57.727072 9593 ts_tablet_manager.cc:1428] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.727133 9592 raft_consensus.cc:348] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.727197 9592 raft_consensus.cc:374] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.727216 9592 raft_consensus.cc:729] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c87f133eb448c2b49a5b7a6c311f21, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.727252 9592 consensus_queue.cc:260] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.727329 9592 ts_tablet_manager.cc:1428] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.727352 9594 tablet_bootstrap.cc:492] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: No bootstrap required, opened a new log
I20260923 22:12:57.727391 9594 ts_tablet_manager.cc:1397] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:12:57.727481 9594 raft_consensus.cc:348] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.727550 9594 raft_consensus.cc:374] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:57.727569 9594 raft_consensus.cc:729] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Initialized, Role: FOLLOWER
I20260923 22:12:57.727623 9594 consensus_queue.cc:260] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.727708 9594 ts_tablet_manager.cc:1428] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:57.846340 9600 raft_consensus.cc:482] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:12:57.846443 9600 raft_consensus.cc:504] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.846613 9600 leader_election.cc:288] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.846835 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42fd00f95ec74673861f884373791098" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" is_pre_election: true
I20260923 22:12:57.846835 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42fd00f95ec74673861f884373791098" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572" is_pre_election: true
I20260923 22:12:57.846923 9518 raft_consensus.cc:2454] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.846930 9378 raft_consensus.cc:2454] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 0.
I20260923 22:12:57.847069 9172 leader_election.cc:302] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, d5c87f133eb448c2b49a5b7a6c311f21; no voters:
I20260923 22:12:57.847170 9600 raft_consensus.cc:2781] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:12:57.847220 9600 raft_consensus.cc:482] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:12:57.847240 9600 raft_consensus.cc:3037] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.847915 9600 raft_consensus.cc:504] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.848043 9600 leader_election.cc:288] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Requested vote from peers c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757), d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657)
I20260923 22:12:57.848234 9378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42fd00f95ec74673861f884373791098" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12eaa35664c443eac0d2e552c415572"
I20260923 22:12:57.848270 9518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42fd00f95ec74673861f884373791098" candidate_uuid: "378bf167f63c4002ad2d2abe7079c37a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c87f133eb448c2b49a5b7a6c311f21"
I20260923 22:12:57.848320 9378 raft_consensus.cc:3037] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.848343 9518 raft_consensus.cc:3037] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:57.848923 9518 raft_consensus.cc:2454] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.848948 9378 raft_consensus.cc:2454] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378bf167f63c4002ad2d2abe7079c37a in term 1.
I20260923 22:12:57.849107 9169 leader_election.cc:302] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378bf167f63c4002ad2d2abe7079c37a, c12eaa35664c443eac0d2e552c415572; no voters:
I20260923 22:12:57.849205 9600 raft_consensus.cc:2781] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:57.849257 9600 raft_consensus.cc:686] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Becoming Leader. State: Replica: 378bf167f63c4002ad2d2abe7079c37a, State: Running, Role: LEADER
I20260923 22:12:57.849306 9600 consensus_queue.cc:237] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } }
I20260923 22:12:57.849942 9079 catalog_manager.cc:5495] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a reported cstate change: term changed from 0 to 1, leader changed from <none> to 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129). New cstate: current_term: 1 leader_uuid: "378bf167f63c4002ad2d2abe7079c37a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378bf167f63c4002ad2d2abe7079c37a" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 32815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 } health_report { overall_health: UNKNOWN } } }
I20260923 22:12:57.875974 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.874010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.1.132.129:32815)
0923 22:12:57.874057 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:57.874059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:57.874067 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:57.874111 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:57.874112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:57.874123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:57.874166 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:57.874246 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.874751 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.875440 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:57.875654 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:57.875704 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:57.875803 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:57.875828 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:57.875859 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:57.875899 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:12:57.875917 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:12:57.878038 9518 raft_consensus.cc:1261] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.878038 9378 raft_consensus.cc:1261] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Refusing update from remote peer 378bf167f63c4002ad2d2abe7079c37a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:57.878257 9610 consensus_queue.cc:1035] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c87f133eb448c2b49a5b7a6c311f21" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.878392 9600 consensus_queue.cc:1035] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12eaa35664c443eac0d2e552c415572" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:57.887804 9080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.887915 9080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:12:57.888563 9080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ced7e076bf34c7ab96f254ece3c6640: Sending DeleteTablet for 3 replicas of tablet 42fd00f95ec74673861f884373791098
I20260923 22:12:57.888835 9218 tablet_service.cc:1514] Processing DeleteTablet for tablet 42fd00f95ec74673861f884373791098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:34244
I20260923 22:12:57.888861 9498 tablet_service.cc:1514] Processing DeleteTablet for tablet 42fd00f95ec74673861f884373791098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36976
I20260923 22:12:57.888949 9616 tablet_replica.cc:330] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: stopping tablet replica
I20260923 22:12:57.888939 9358 tablet_service.cc:1514] Processing DeleteTablet for tablet 42fd00f95ec74673861f884373791098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:12:57 UTC) from {username='oryx', principal='oryx/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51796
I20260923 22:12:57.888995 9616 raft_consensus.cc:2229] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.889029 9616 raft_consensus.cc:2258] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.889022 9615 tablet_replica.cc:330] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a: stopping tablet replica
I20260923 22:12:57.889065 9615 raft_consensus.cc:2229] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:12:57.889117 9615 raft_consensus.cc:2258] T 42fd00f95ec74673861f884373791098 P 378bf167f63c4002ad2d2abe7079c37a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.889181 9617 tablet_replica.cc:330] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: stopping tablet replica
I20260923 22:12:57.889181 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9153
I20260923 22:12:57.889233 9617 raft_consensus.cc:2229] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:12:57.889269 9617 raft_consensus.cc:2258] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:12:57.889904 9616 ts_tablet_manager.cc:1905] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.890908 9617 ts_tablet_manager.cc:1905] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:12:57.891458 9616 ts_tablet_manager.cc:1918] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:57.891507 9616 log.cc:1196] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/42fd00f95ec74673861f884373791098
I20260923 22:12:57.891685 9616 ts_tablet_manager.cc:1939] T 42fd00f95ec74673861f884373791098 P d5c87f133eb448c2b49a5b7a6c311f21: Deleting consensus metadata
I20260923 22:12:57.892067 9068 catalog_manager.cc:4841] TS d5c87f133eb448c2b49a5b7a6c311f21 (127.1.132.131:43657): tablet 42fd00f95ec74673861f884373791098 (table test-table [id=df7dbb01aecc48bbad52328799f93056]) successfully deleted
I20260923 22:12:57.892114 9617 ts_tablet_manager.cc:1918] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:12:57.892154 9617 log.cc:1196] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/42fd00f95ec74673861f884373791098
I20260923 22:12:57.892341 9617 ts_tablet_manager.cc:1939] T 42fd00f95ec74673861f884373791098 P c12eaa35664c443eac0d2e552c415572: Deleting consensus metadata
I20260923 22:12:57.892769 9065 catalog_manager.cc:4841] TS c12eaa35664c443eac0d2e552c415572 (127.1.132.130:36757): tablet 42fd00f95ec74673861f884373791098 (table test-table [id=df7dbb01aecc48bbad52328799f93056]) successfully deleted
W20260923 22:12:57.896055 9067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:32815 (error 108)
I20260923 22:12:57.896593 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:57.896416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:32815 (local address 127.0.0.1:34274)
0923 22:12:57.896507 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:57.896549 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:32815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":40,"threads_started":1}
I20260923 22:12:57.896695 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9293
W20260923 22:12:57.896734 9067 catalog_manager.cc:4584] TS 378bf167f63c4002ad2d2abe7079c37a (127.1.132.129:32815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42fd00f95ec74673861f884373791098: Network error: Client connection negotiation failed: client connection to 127.1.132.129:32815: connect: Connection refused (error 111)
I20260923 22:12:57.901494 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9433
I20260923 22:12:57.906703 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9049
2026-09-23T22:12:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2673 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9678](info): set up 2 sockets
Sep 23 22:12:57 dist-test-slave-l4q5 krb5kdc[9678](info): commencing operation
krb5kdc: starting...
W20260923 22:12:58.872570 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:12:58 dist-test-slave-l4q5 krb5kdc[9678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:12:58.996711 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39877
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:39877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:59.079993 9691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:59.080245 9691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:59.080268 9691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:59.081506 9691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:59.081560 9691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:59.081574 9691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:59.081585 9691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:59.081596 9691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:59.083173 9691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:39877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:12:59.083410 9691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:59.083639 9691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:59.088425 9697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:59.088444 9696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:59.088565 9691 server_base.cc:1034] running on GCE node
W20260923 22:12:59.088420 9699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:59.088876 9691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:59.089186 9691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:59.090346 9691 hybrid_clock.cc:648] HybridClock initialized: now 1790201579090332 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T22:12:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1188 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:59 dist-test-slave-l4q5 krb5kdc[9704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:59 dist-test-slave-l4q5 krb5kdc[9704](info): set up 2 sockets
Sep 23 22:12:59 dist-test-slave-l4q5 krb5kdc[9704](info): commencing operation
krb5kdc: starting...
W20260923 22:13:00.060961 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:13:00.190315 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40917
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:00.276144 9717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:00.276419 9717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:00.276445 9717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:00.277806 9717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:00.277870 9717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:00.277889 9717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:00.277901 9717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:00.277912 9717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:00.279647 9717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:00.279914 9717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:00.280144 9717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:00.285825 9723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.285806 9725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.285799 9722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:00.286144 9717 server_base.cc:1034] running on GCE node
I20260923 22:13:00.286401 9717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:00.286721 9717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:00.287894 9717 hybrid_clock.cc:648] HybridClock initialized: now 1790201580287877 us; error 28 us; skew 500 ppm
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:00.299259 9717 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:13:00.299893 9717 webserver.cc:460] Webserver started at http://127.1.132.190:43097/ using document root <none> and password file <none>
I20260923 22:13:00.300137 9717 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:00.300230 9717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:00.300380 9717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:00.301542 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "9f8677dbcf8346ee8b3fa089b3241e04"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "99e8db8bbcee91b8ccd6db7e727cfd0f"
server_key_iv: "0d4c35c382a881da074edd5627e1b2d5"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.301914 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "9f8677dbcf8346ee8b3fa089b3241e04"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "99e8db8bbcee91b8ccd6db7e727cfd0f"
server_key_iv: "0d4c35c382a881da074edd5627e1b2d5"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.303382 9717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:13:00.304349 9732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.304764 9717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:00.304996 9717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "9f8677dbcf8346ee8b3fa089b3241e04"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "99e8db8bbcee91b8ccd6db7e727cfd0f"
server_key_iv: "0d4c35c382a881da074edd5627e1b2d5"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.305110 9717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:00.319334 9717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:00.321731 9717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:00.321878 9717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:00.326206 9717 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40917
I20260923 22:13:00.326229 9784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40917 every 8 connection(s)
I20260923 22:13:00.326622 9717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:00.327575 9785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:00.330016 9785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Bootstrap starting.
I20260923 22:13:00.330559 9785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:00.330808 9785 log.cc:824] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:00.331444 9785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: No bootstrap required, opened a new log
I20260923 22:13:00.333130 9785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:00.333285 9785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:00.333308 9785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8677dbcf8346ee8b3fa089b3241e04, State: Initialized, Role: FOLLOWER
I20260923 22:13:00.333412 9785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:00.333463 9785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:00.333484 9785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:00.333529 9785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:00.334088 9785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:00.334178 9785 leader_election.cc:302] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f8677dbcf8346ee8b3fa089b3241e04; no voters:
I20260923 22:13:00.334343 9785 leader_election.cc:288] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:00.334393 9788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:00.334612 9785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:00.334774 9788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 LEADER]: Becoming Leader. State: Replica: 9f8677dbcf8346ee8b3fa089b3241e04, State: Running, Role: LEADER
I20260923 22:13:00.334873 9788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:00.335242 9789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } } }
I20260923 22:13:00.335254 9790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f8677dbcf8346ee8b3fa089b3241e04. Latest consensus state: current_term: 1 leader_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } } }
I20260923 22:13:00.335345 9789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:00.335359 9790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:00.335546 9797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:00.335858 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9717
I20260923 22:13:00.335968 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:13:00.336023 9797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:00.336200 1554 external_mini_cluster.cc:1428] Setting key b3c2f1a196c4bb92e6fcf1545856d725
I20260923 22:13:00.337850 9797 catalog_manager.cc:1347] Generated new cluster ID: a130cc06851d480fb9dd5047d41c230a
I20260923 22:13:00.337895 9797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:00.346786 9797 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:00.347471 9797 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:00.348630 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.336820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.1.132.190:40917)
0923 22:13:00.336954 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.336963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.336996 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:13:00.337107 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.337114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.337179 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.337314 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.338050 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.339497 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.340786 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.341484 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.341612 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.344539 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:00.344545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:00.344546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:00.344562 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:00.346590 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.347044 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.347048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.347048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.347174 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.347360 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.347361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.347361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.347619 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:00.347778 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:00.348229 (+ 451us) server_negotiation.cc:299] Negotiation successful
0923 22:13:00.348386 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
I20260923 22:13:00.352905 9797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Generated new TSK 0
I20260923 22:13:00.353084 9797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:13:00.384405 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:00.464566 9811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:00.464780 9811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:00.464804 9811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:00.466061 9811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:00.466110 9811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:00.466145 9811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:13:00.467643 9811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:00.467871 9811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:00.468101 9811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:00.472998 9817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:00.473153 9811 server_base.cc:1034] running on GCE node
W20260923 22:13:00.472997 9819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.472985 9816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:00.473453 9811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:00.473691 9811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:00.474849 9811 hybrid_clock.cc:648] HybridClock initialized: now 1790201580474838 us; error 30 us; skew 500 ppm
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:00.477334 9811 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:13:00.477777 9811 webserver.cc:460] Webserver started at http://127.1.132.129:41131/ using document root <none> and password file <none>
I20260923 22:13:00.477993 9811 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:00.478040 9811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:00.478173 9811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:00.479223 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "d7f82534daea4cc2b92a748b99e7320d"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "3964a16de3f8d67ba153dcb66c4fd216"
server_key_iv: "30e01a3c164b9730a61c3cb420858aa3"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.479552 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "d7f82534daea4cc2b92a748b99e7320d"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "3964a16de3f8d67ba153dcb66c4fd216"
server_key_iv: "30e01a3c164b9730a61c3cb420858aa3"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.480921 9811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:00.481673 9826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.481848 9811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:13:00.481945 9811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "d7f82534daea4cc2b92a748b99e7320d"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "3964a16de3f8d67ba153dcb66c4fd216"
server_key_iv: "30e01a3c164b9730a61c3cb420858aa3"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.482014 9811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:00.497521 9811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:00.499624 9811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:00.499770 9811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:00.500002 9811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:00.500383 9811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:00.500420 9811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.500449 9811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:00.500473 9811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.505993 9811 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:36131
I20260923 22:13:00.506038 9939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:36131 every 8 connection(s)
I20260923 22:13:00.506348 9811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:00.509364 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9811
I20260923 22:13:00.509465 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:00.509697 1554 external_mini_cluster.cc:1428] Setting key 134e8b47c9d2fc518b79f69c4665f83c
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:00.516247 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.507072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:49563 (local address 127.1.132.190:40917)
0923 22:13:00.507139 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.507141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.507323 (+ 182us) server_negotiation.cc:407] Connection header received
0923 22:13:00.507835 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.507836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.507846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.507914 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.508892 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.509438 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.510739 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.510939 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.510992 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.513312 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:00.513314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:00.513315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:00.513321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:00.514677 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.515169 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.515170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.515171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.515230 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.515507 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.515507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.515508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.515662 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:00.515719 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:00.516064 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 22:13:00.516145 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:13:00.516479 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.507031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:49563)
0923 22:13:00.507195 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.507240 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.507742 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.507968 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.507977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.508602 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.508814 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.508819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.509538 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.509540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.510666 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.510669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.511014 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.511015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.511179 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.511704 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.511714 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.513200 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:00.514756 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.514765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.514768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.515069 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.515298 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.515298 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.515299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.515438 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.515728 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:00.515735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:00.515910 (+ 175us) client_negotiation.cc:769] Sending connection context
0923 22:13:00.516030 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:13:00.516243 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260923 22:13:00.516992 9940 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40917
I20260923 22:13:00.517112 9940 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:00.517294 9940 heartbeater.cc:511] Master 127.1.132.190:40917 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:13:00.517765 9749 ts_manager.cc:194] Registered new tserver with Master: d7f82534daea4cc2b92a748b99e7320d (127.1.132.129:36131)
I20260923 22:13:00.518338 9749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:49563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:13:00.535588 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:00.617774 9945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:00.618036 9945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:00.618072 9945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:00.619730 9945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:00.619807 9945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:00.619863 9945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:13:00.622036 9945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:00.622331 9945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:00.622573 9945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:00.627579 9950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.627738 9951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.627640 9953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:00.627897 9945 server_base.cc:1034] running on GCE node
I20260923 22:13:00.628145 9945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:00.628448 9945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:00.629652 9945 hybrid_clock.cc:648] HybridClock initialized: now 1790201580629637 us; error 27 us; skew 500 ppm
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:00.632265 9945 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:13:00.632802 9945 webserver.cc:460] Webserver started at http://127.1.132.130:42545/ using document root <none> and password file <none>
I20260923 22:13:00.633023 9945 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:00.633072 9945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:00.633203 9945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:00.634258 9945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "1ace41d3bbbd455894437ab014971ad5"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "f4e2ca50d397964898869e6baab88408"
server_key_iv: "5b980e28b76d7988fb56eb6aa47a35d6"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.634613 9945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "1ace41d3bbbd455894437ab014971ad5"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "f4e2ca50d397964898869e6baab88408"
server_key_iv: "5b980e28b76d7988fb56eb6aa47a35d6"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.636072 9945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:00.636969 9960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.637157 9945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:00.637251 9945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "1ace41d3bbbd455894437ab014971ad5"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "f4e2ca50d397964898869e6baab88408"
server_key_iv: "5b980e28b76d7988fb56eb6aa47a35d6"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.637323 9945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:00.648892 9945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:00.651053 9945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:00.651197 9945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:00.651422 9945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:00.651752 9945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:00.651788 9945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.651819 9945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:00.651836 9945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.658002 9945 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:33095
I20260923 22:13:00.658053 10073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:33095 every 8 connection(s)
I20260923 22:13:00.658385 9945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:00.660689 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 9945
I20260923 22:13:00.660809 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:00.661002 1554 external_mini_cluster.cc:1428] Setting key dec8e07af9bdbc62b2acb4418092ae22
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:13:00.669562 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.659126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:51669 (local address 127.1.132.190:40917)
0923 22:13:00.659217 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.659220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.659425 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:13:00.660124 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.660126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.660136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.660260 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.661501 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.662072 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.663404 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.663756 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.663835 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.666334 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:00.666336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:00.666337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:00.666346 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:00.668042 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.668515 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.668517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.668518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.668585 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.668840 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.668841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.668842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.668991 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:00.669072 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:00.669388 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 22:13:00.669470 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 22:13:00.669835 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.659044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:51669)
0923 22:13:00.659301 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.659351 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.659995 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.660353 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.660357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.661164 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.661391 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.661397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.662164 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.662165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.663305 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.663308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.663866 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.663867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.663991 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.664554 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.664563 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.666218 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:00.668157 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.668164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.668166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.668434 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.668660 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.668660 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.668661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.668774 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.669095 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:00.669104 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:00.669247 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:13:00.669347 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:13:00.669532 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260923 22:13:00.670384 10074 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40917
I20260923 22:13:00.670481 10074 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:00.670658 10074 heartbeater.cc:511] Master 127.1.132.190:40917 requested a full tablet report, sending...
I20260923 22:13:00.671079 9749 ts_manager.cc:194] Registered new tserver with Master: 1ace41d3bbbd455894437ab014971ad5 (127.1.132.130:33095)
I20260923 22:13:00.671481 9749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:51669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:13:00.691977 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:00.775316 10079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:00.775552 10079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:00.775578 10079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:00.776922 10079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:00.776973 10079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:00.777002 10079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:13:00.778743 10079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:00.779378 10079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:00.779657 10079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:00.785220 10087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.785229 10085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:00.785459 10084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:00.785597 10079 server_base.cc:1034] running on GCE node
I20260923 22:13:00.785811 10079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:00.786048 10079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:00.787213 10079 hybrid_clock.cc:648] HybridClock initialized: now 1790201580787198 us; error 31 us; skew 500 ppm
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:00.789723 10079 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:13:00.790128 10079 webserver.cc:460] Webserver started at http://127.1.132.131:45151/ using document root <none> and password file <none>
I20260923 22:13:00.790351 10079 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:00.790409 10079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:00.790527 10079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:00.791630 10079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "2b315c3e39594e5ba29a583ddcb39989"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "b784c4683956be58c7d8601dc892419e"
server_key_iv: "aa32947df264c4cf8c3f415b8ddd7c07"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.791980 10079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "2b315c3e39594e5ba29a583ddcb39989"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "b784c4683956be58c7d8601dc892419e"
server_key_iv: "aa32947df264c4cf8c3f415b8ddd7c07"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.793365 10079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:00.794239 10094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.794453 10079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:00.794545 10079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "2b315c3e39594e5ba29a583ddcb39989"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "b784c4683956be58c7d8601dc892419e"
server_key_iv: "aa32947df264c4cf8c3f415b8ddd7c07"
server_key_version: "encryptionkey@0"
I20260923 22:13:00.794624 10079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:00.810763 10079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:00.813263 10079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:00.813382 10079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:00.813571 10079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:00.813985 10079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:00.814037 10079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.814071 10079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:00.814088 10079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:00.820047 10079 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:40829
I20260923 22:13:00.820097 10207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:40829 every 8 connection(s)
I20260923 22:13:00.820443 10079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:13:00 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201580, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:00.827085 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10079
I20260923 22:13:00.827211 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:00.827394 1554 external_mini_cluster.cc:1428] Setting key 9daeee42137c9472edf24a37e2b86bb4
I20260923 22:13:00.829747 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.821277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:41991 (local address 127.1.132.190:40917)
0923 22:13:00.821336 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:00.821338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:00.821548 (+ 210us) server_negotiation.cc:407] Connection header received
0923 22:13:00.822067 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:00.822068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:00.822081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:00.822134 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:00.823081 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.823576 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.824598 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.824787 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.824840 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.826863 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:00.826865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:00.826865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:00.826873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:00.828388 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.828942 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.828943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.828943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.829004 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:00.829240 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:00.829240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:00.829240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:00.829347 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:00.829394 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:00.829603 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 22:13:00.829673 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:13:00.830052 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:00.821230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:41991)
0923 22:13:00.821424 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:00.821462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:00.821963 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:00.822216 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:00.822220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:00.822787 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:00.822981 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.822986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.823669 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.823670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.824503 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:00.824506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:00.824856 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:00.824857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:00.824978 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:00.825443 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:00.825450 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:00.826769 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:00.828527 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.828551 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.828553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.828843 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.829076 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:00.829077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:00.829077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:00.829188 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:00.829385 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:00.829388 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:00.829506 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 22:13:00.829584 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:13:00.829746 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260923 22:13:00.830531 10208 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40917
I20260923 22:13:00.830627 10208 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:00.830811 10208 heartbeater.cc:511] Master 127.1.132.190:40917 requested a full tablet report, sending...
I20260923 22:13:00.831275 9749 ts_manager.cc:194] Registered new tserver with Master: 2b315c3e39594e5ba29a583ddcb39989 (127.1.132.131:40829)
I20260923 22:13:00.831789 9749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:41991
I20260923 22:13:00.839650 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
Entry for principal oryx/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab.
WARNING: no policy specified for oryx/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
Entry for principal oryx/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab.
WARNING: no policy specified for oryx/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
Entry for principal oryx/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab.
I20260923 22:13:00.915561 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9811
I20260923 22:13:00.919631 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9945
I20260923 22:13:00.923636 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10079
I20260923 22:13:00.927629 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 9717
I20260923 22:13:00.932219 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40917
--webserver_interface=127.1.132.190
--webserver_port=43097
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:01.013768 10217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:01.014012 10217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:01.014056 10217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:01.015362 10217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:01.015417 10217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:01.015431 10217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:01.015442 10217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:01.015453 10217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:01.017115 10217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=43097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:01.017374 10217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:01.017598 10217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:01.022644 10222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.022709 10225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.022711 10223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:01.022763 10217 server_base.cc:1034] running on GCE node
I20260923 22:13:01.023088 10217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:01.023368 10217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:01.024531 10217 hybrid_clock.cc:648] HybridClock initialized: now 1790201581024499 us; error 46 us; skew 500 ppm
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201581, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:01.027204 10217 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:13:01.027716 10217 webserver.cc:460] Webserver started at http://127.1.132.190:43097/ using document root <none> and password file <none>
I20260923 22:13:01.027926 10217 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:01.027984 10217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:01.029711 10217 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:01.030449 10232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.030633 10217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:01.030761 10217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "9f8677dbcf8346ee8b3fa089b3241e04"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "99e8db8bbcee91b8ccd6db7e727cfd0f"
server_key_iv: "0d4c35c382a881da074edd5627e1b2d5"
server_key_version: "encryptionkey@0"
I20260923 22:13:01.031232 10217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:01.050503 10217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:01.052676 10217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:01.052827 10217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:01.056911 10217 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40917
I20260923 22:13:01.056962 10284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40917 every 8 connection(s)
I20260923 22:13:01.057337 10217 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:01.058770 10285 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:13:01.059396 10285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Bootstrap starting.
I20260923 22:13:01.067178 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10217
I20260923 22:13:01.067525 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:36131
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:40917
--webserver_port=41131
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:13:01.067785 10285 log.cc:824] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:01.069733 10285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:13:01.070007 10285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Bootstrap complete.
I20260923 22:13:01.072206 10285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:01.072485 10285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8677dbcf8346ee8b3fa089b3241e04, State: Initialized, Role: FOLLOWER
I20260923 22:13:01.072636 10285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:01.072705 10285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:01.072744 10285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:01.072795 10285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:13:01.073846 10285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:01.073940 10285 leader_election.cc:302] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f8677dbcf8346ee8b3fa089b3241e04; no voters:
I20260923 22:13:01.074113 10285 leader_election.cc:288] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:13:01.074163 10289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:13:01.074308 10289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [term 2 LEADER]: Becoming Leader. State: Replica: 9f8677dbcf8346ee8b3fa089b3241e04, State: Running, Role: LEADER
I20260923 22:13:01.074401 10285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:01.074458 10289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } }
I20260923 22:13:01.074780 10290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } } }
I20260923 22:13:01.074919 10290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:01.075186 10298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:01.075196 10291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f8677dbcf8346ee8b3fa089b3241e04. Latest consensus state: current_term: 2 leader_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f8677dbcf8346ee8b3fa089b3241e04" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40917 } } }
I20260923 22:13:01.075376 10291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:01.075795 10298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:01.075982 10298 catalog_manager.cc:1259] Loaded cluster ID: a130cc06851d480fb9dd5047d41c230a
I20260923 22:13:01.076162 10298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:01.076969 10298 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:01.077124 10298 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f8677dbcf8346ee8b3fa089b3241e04: Loaded TSK: 0
I20260923 22:13:01.077320 10298 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:13:01.171309 10288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:01.171525 10288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:01.171568 10288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:01.172940 10288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:01.172992 10288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:01.173029 10288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:13:01.174666 10288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.129.keytab
--principal=oryx/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:36131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=41131
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:01.174979 10288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:01.175235 10288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:01.180388 10314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.180390 10312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.180437 10311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:01.180932 10288 server_base.cc:1034] running on GCE node
I20260923 22:13:01.181180 10288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:01.181427 10288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:01.182583 10288 hybrid_clock.cc:648] HybridClock initialized: now 1790201581182574 us; error 31 us; skew 500 ppm
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:01.185213 10288 init.cc:377] Logged in from keytab as oryx/127.1.132.129@KRBTEST.COM (short username oryx)
I20260923 22:13:01.185750 10288 webserver.cc:460] Webserver started at http://127.1.132.129:41131/ using document root <none> and password file <none>
I20260923 22:13:01.185967 10288 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:01.186025 10288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:01.187417 10288 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:01.188099 10321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.188292 10288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:01.188386 10288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "d7f82534daea4cc2b92a748b99e7320d"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "3964a16de3f8d67ba153dcb66c4fd216"
server_key_iv: "30e01a3c164b9730a61c3cb420858aa3"
server_key_version: "encryptionkey@0"
I20260923 22:13:01.188776 10288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:01.211755 10288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:01.214227 10288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:01.214375 10288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:01.214708 10288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:01.215086 10288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:01.215137 10288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.215170 10288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:01.215194 10288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.222791 10288 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:36131
I20260923 22:13:01.222869 10434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:36131 every 8 connection(s)
I20260923 22:13:01.223238 10288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.233270 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10288
I20260923 22:13:01.233574 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:33095
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:40917
--webserver_port=42545
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:01.233708 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.224025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:38681 (local address 127.1.132.190:40917)
0923 22:13:01.224282 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.224290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.224320 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:13:01.225093 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.225099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.225145 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.225258 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.226513 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.227464 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.229225 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.229764 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.229865 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.233527 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:38681: BlockingRecv error: recv got EOF from 127.1.132.129:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260923 22:13:01.233716 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.223927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:38681)
0923 22:13:01.224095 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.224160 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.224860 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.225333 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.225337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.226096 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.226374 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.226381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.227561 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.227561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.229043 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.229048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.229870 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.229871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.230028 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.230627 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.230640 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.233392 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260923 22:13:01.234086 10435 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40917 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.132.190:40917: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.238591 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.234333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:51999)
0923 22:13:01.234382 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.234392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.234429 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.234603 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.234604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.234738 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.234787 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.234789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.235436 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.235437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.235949 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.235950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.236321 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.236321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.236388 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.236827 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.236832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.238513 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:13:01.238731 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.234426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51999 (local address 127.1.132.190:40917)
0923 22:13:01.234493 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.234495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.234502 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:01.234527 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.234528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.234539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.234590 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.234867 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.235365 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.236033 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.236264 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.236321 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.238621 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:51999: BlockingRecv error: recv got EOF from 127.1.132.129:51999 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.243767 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.238877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:40709)
0923 22:13:01.238977 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.238989 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.239037 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.239180 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.239181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.239325 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.239369 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.239371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.240030 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.240031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.240851 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.240855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.241222 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.241222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.241278 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.241708 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.241714 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.243693 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260923 22:13:01.243925 10435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:13:01.244019 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.238973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:40709 (local address 127.1.132.190:40917)
0923 22:13:01.239020 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.239022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.239031 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:01.239101 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.239102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.239110 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.239160 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.239433 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.239953 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.240948 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.241141 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.241204 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.243800 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:40709: BlockingRecv error: recv got EOF from 127.1.132.129:40709 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260923 22:13:01.318977 10439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:01.319178 10439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:01.319202 10439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:01.320374 10439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:01.320410 10439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:01.320436 10439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:13:01.321822 10439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.130.keytab
--principal=oryx/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:33095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=42545
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:01.322014 10439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:01.322197 10439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:01.326992 10445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.327052 10444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.327128 10447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:01.327255 10439 server_base.cc:1034] running on GCE node
I20260923 22:13:01.327451 10439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:01.327693 10439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:01.328836 10439 hybrid_clock.cc:648] HybridClock initialized: now 1790201581328824 us; error 41 us; skew 500 ppm
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:01.331152 10439 init.cc:377] Logged in from keytab as oryx/127.1.132.130@KRBTEST.COM (short username oryx)
I20260923 22:13:01.331558 10439 webserver.cc:460] Webserver started at http://127.1.132.130:42545/ using document root <none> and password file <none>
I20260923 22:13:01.331765 10439 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:01.331820 10439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:01.333125 10439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:01.333781 10454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.334023 10439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:01.334107 10439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "1ace41d3bbbd455894437ab014971ad5"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "f4e2ca50d397964898869e6baab88408"
server_key_iv: "5b980e28b76d7988fb56eb6aa47a35d6"
server_key_version: "encryptionkey@0"
I20260923 22:13:01.334481 10439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:01.358507 10439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:01.360775 10439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:01.360924 10439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:01.361155 10439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:01.361467 10439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:01.361500 10439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.361528 10439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:01.361550 10439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.366808 10439 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:33095
I20260923 22:13:01.366869 10567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:33095 every 8 connection(s)
I20260923 22:13:01.367194 10439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:01.368813 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10439
I20260923 22:13:01.369073 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:40829
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:40917
--webserver_port=45151
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:13:01.375562 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.367929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:42683 (local address 127.1.132.190:40917)
0923 22:13:01.368054 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.368056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.368332 (+ 276us) server_negotiation.cc:407] Connection header received
0923 22:13:01.368856 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.368858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.368873 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.368924 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.369889 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.370388 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.371542 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.371852 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.371902 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.375503 (+ 3601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:42683: BlockingRecv error: recv got EOF from 127.1.132.130:42683 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260923 22:13:01.375712 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.367919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:42683)
0923 22:13:01.368177 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.368247 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.368757 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.368995 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.368999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.369587 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.369792 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.369798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.370474 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.370475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.371410 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.371414 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.372002 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.372003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.372160 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.372832 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.372842 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.375409 (+ 2567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":161,"threads_started":1}
W20260923 22:13:01.376027 10568 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40917 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.132.190:40917: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.381070 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.376327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:57181)
0923 22:13:01.376414 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.376424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.376476 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.376607 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.376608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.376758 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.376831 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.376833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.377472 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.377473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.378292 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.378296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.378630 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.378631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.378702 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.379109 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.379114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.380982 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260923 22:13:01.381143 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.376362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:57181 (local address 127.1.132.190:40917)
0923 22:13:01.376403 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.376405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.376460 (+ 55us) server_negotiation.cc:407] Connection header received
0923 22:13:01.376540 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.376541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.376548 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.376597 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.376907 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.377395 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.378377 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.378560 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.378611 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.381085 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:57181: BlockingRecv error: recv got EOF from 127.1.132.130:57181 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.385807 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.381518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:33625)
0923 22:13:01.381596 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.381607 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.381653 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.381762 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.381763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.381865 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.381896 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.381897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.382799 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.382800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.383314 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.383315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.383762 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.383763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.383809 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.384266 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.384271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.385754 (+ 1483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:13:01.385963 10568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:13:01.385987 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.381570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:33625 (local address 127.1.132.190:40917)
0923 22:13:01.381610 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.381612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.381642 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:13:01.381694 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.381695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.381704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.381749 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.381944 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.382721 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.383377 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.383674 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.383735 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.385921 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:33625: BlockingRecv error: recv got EOF from 127.1.132.130:33625 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:13:01.452379 10571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:01.452595 10571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:01.452618 10571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:01.453795 10571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:01.453832 10571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:01.453857 10571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:13:01.455526 10571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/krb5kdc/oryx_127.1.132.131.keytab
--principal=oryx/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:40829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=45151
--tserver_master_addrs=127.1.132.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:01.455746 10571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:01.455940 10571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:01.461079 10577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.461063 10576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:01.461063 10579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:01.461341 10571 server_base.cc:1034] running on GCE node
I20260923 22:13:01.461575 10571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:01.461812 10571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:01.462983 10571 hybrid_clock.cc:648] HybridClock initialized: now 1790201581462967 us; error 32 us; skew 500 ppm
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:01.465622 10571 init.cc:377] Logged in from keytab as oryx/127.1.132.131@KRBTEST.COM (short username oryx)
I20260923 22:13:01.466064 10571 webserver.cc:460] Webserver started at http://127.1.132.131:45151/ using document root <none> and password file <none>
I20260923 22:13:01.466274 10571 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:01.466328 10571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:01.467818 10571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:01.468580 10586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.468818 10571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:13:01.468916 10571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "2b315c3e39594e5ba29a583ddcb39989"
format_stamp: "Formatted at 2026-09-23 22:13:00 on dist-test-slave-l4q5"
server_key: "b784c4683956be58c7d8601dc892419e"
server_key_iv: "aa32947df264c4cf8c3f415b8ddd7c07"
server_key_version: "encryptionkey@0"
I20260923 22:13:01.469290 10571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:01.483140 10571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:01.485188 10571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:01.485304 10571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:01.485488 10571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:01.485777 10571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:01.485808 10571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.485837 10571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:01.485860 10571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:01.491603 10571 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:40829
I20260923 22:13:01.491665 10699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:40829 every 8 connection(s)
I20260923 22:13:01.492014 10571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:01.494001 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10571
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:13:01.500059 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.492721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:52355 (local address 127.1.132.190:40917)
0923 22:13:01.492789 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.492792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.492994 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:13:01.493449 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.493450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.493460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.493508 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.494641 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.495381 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.496752 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.497057 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.497137 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.500011 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:52355: BlockingRecv error: recv got EOF from 127.1.132.131:52355 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:13:01.500093 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.492722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:52355)
0923 22:13:01.492878 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.492916 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.493375 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.493560 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.493563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.494257 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.494526 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.494533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.495476 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.495477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.496670 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.496673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.497153 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.497154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.497301 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.497875 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.497883 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.499932 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260923 22:13:01.500342 10700 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40917 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.132.190:40917: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
I20260923 22:13:01.502234 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.494499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.1.132.190:40917)
0923 22:13:01.494620 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.494622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.494629 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:01.494650 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.494651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.494663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.494721 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.495416 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.496223 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.497140 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.497419 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.497489 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.498607 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:01.498612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:01.498612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:01.498624 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:01.500978 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:01.501326 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:01.501329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:01.501329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:01.501415 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:01.501574 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:01.501574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:01.501575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:01.501768 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:01.501859 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:01.502040 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 22:13:01.502131 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.504948 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.500545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:60605)
0923 22:13:01.500588 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.500597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.500640 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.500760 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.500761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.500886 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.500930 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.500932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.501717 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.501718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.502438 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.502441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.502861 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.502862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.502929 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.503292 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.503296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.504877 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:13:01.505018 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.500600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:60605 (local address 127.1.132.190:40917)
0923 22:13:01.500638 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.500640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.500647 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:01.500703 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.500703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.500713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.500758 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.500994 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.501654 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.502522 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.502752 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.502795 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.504969 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:60605: BlockingRecv error: recv got EOF from 127.1.132.131:60605 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:01 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:01.509330 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:01.505261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:57125)
0923 22:13:01.505303 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:01.505312 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:01.505349 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:01.505443 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:01.505444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:01.505560 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:01.505588 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.505589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.506175 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.506176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.506664 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:01.506665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.506972 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.506973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:01.507024 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.507371 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:01.507376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:01.509261 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:13:01.509405 10438 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:01.505314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:57125 (local address 127.1.132.190:40917)
0923 22:13:01.505355 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:01.505357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:01.505366 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:01.505388 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:01.505389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:01.505398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:01.505442 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:01.505651 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.506108 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.506727 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:01.506920 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:01.506969 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:01.509354 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:57125: BlockingRecv error: recv got EOF from 127.1.132.131:57125 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:13:01.509505 10700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.249034 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.244355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:44103)
0923 22:13:02.244507 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.244519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.244588 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.244720 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.244721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.244877 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.244926 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.244927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.245616 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.245617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.246177 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.246179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.246585 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.246587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.246641 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.247073 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.247078 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.248944 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":60,"threads_started":1}
W20260923 22:13:02.249109 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.244466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:44103 (local address 127.1.132.190:40917)
0923 22:13:02.244574 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.244577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.244588 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:13:02.244639 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.244640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.244649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.244707 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.245014 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.245528 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.246279 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.246488 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.246551 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.249047 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:44103: BlockingRecv error: recv got EOF from 127.1.132.129:44103 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":44,"threads_started":1}
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.390928 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.386400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:36821)
0923 22:13:02.386580 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.386592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.386669 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.386794 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.386794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.386964 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.387012 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.387014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.387688 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.387689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.388271 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.388274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.388606 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.388606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.388661 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.389089 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.389094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.390857 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
W20260923 22:13:02.391016 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.386499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:36821 (local address 127.1.132.190:40917)
0923 22:13:02.386555 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.386558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.386653 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:13:02.386734 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.386735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.386746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.386789 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.387108 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.387604 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.388343 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.388542 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.388599 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.390957 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:36821: BlockingRecv error: recv got EOF from 127.1.132.130:36821 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:02 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:02.514858 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:02.509951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:47793)
0923 22:13:02.510146 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:02.510158 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:02.510226 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:02.510348 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:02.510349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:02.510507 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:02.510554 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.510555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.511202 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.511203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.511799 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:02.511801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.512235 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.512237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:02.512306 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.512779 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:02.512785 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:02.514790 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":68,"threads_started":1}
W20260923 22:13:02.514957 10706 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:02.510063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:47793 (local address 127.1.132.190:40917)
0923 22:13:02.510174 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:02.510175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:02.510212 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:13:02.510285 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:02.510286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:02.510300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:02.510342 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:02.510612 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.511102 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.511905 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:02.512097 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:02.512167 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:02.514897 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:47793: BlockingRecv error: recv got EOF from 127.1.132.131:47793 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.254209 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.249630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:49233)
0923 22:13:03.249803 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.249814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.249865 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.250080 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.250080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.250263 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.250312 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.250314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.250983 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.250984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.251539 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.251541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.251903 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.251904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.251952 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.252415 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.252420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.254130 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
W20260923 22:13:03.254288 10710 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.249729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:49233 (local address 127.1.132.190:40917)
0923 22:13:03.249836 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.249837 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.249849 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:13:03.249968 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.249969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.249981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.250050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.250379 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.250898 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.251631 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.251829 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.251891 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.254228 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:49233: BlockingRecv error: recv got EOF from 127.1.132.129:49233 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.396718 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.391409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:36873)
0923 22:13:03.391567 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.391580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.391643 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.391809 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.391811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.391973 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.392021 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.392023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.392879 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.392881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.393416 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.393418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.393919 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.393920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.393982 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.394620 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.394626 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.396654 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":60,"threads_started":1}
W20260923 22:13:03.396857 10710 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.391537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:36873 (local address 127.1.132.190:40917)
0923 22:13:03.391633 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.391635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.391647 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:13:03.391705 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.391706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.391721 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.391765 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.392123 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.392790 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.393522 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.393818 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.393880 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.396795 (+ 2915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:36873: BlockingRecv error: recv got EOF from 127.1.132.130:36873 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:03 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:03.519930 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:03.515382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:39127)
0923 22:13:03.515533 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:03.515543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:03.515604 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:03.515744 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:03.515745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:03.515924 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:03.515969 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.515970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.516770 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.516771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.517290 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:03.517292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.517676 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.517678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:03.517730 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.518189 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:03.518194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:03.519853 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
W20260923 22:13:03.520017 10710 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:03.515555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:39127 (local address 127.1.132.190:40917)
0923 22:13:03.515615 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:03.515617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:03.515628 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:13:03.515677 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:03.515678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:03.515692 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:03.515740 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:03.516048 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.516684 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.517370 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:03.517569 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:03.517622 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:03.519961 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:39127: BlockingRecv error: recv got EOF from 127.1.132.131:39127 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.259689 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.254804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:60757)
0923 22:13:04.254959 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.254970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.255020 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.255223 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.255224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.255387 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.255438 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.255439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.256143 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.256144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.256750 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.256753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.257166 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.257167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.257218 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.257655 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.257660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.259624 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
W20260923 22:13:04.259816 10714 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.254927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:60757 (local address 127.1.132.190:40917)
0923 22:13:04.255043 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.255044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.255076 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:13:04.255100 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.255101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.255110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.255165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.255518 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.256055 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.256851 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.257075 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.257142 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.259736 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:60757: BlockingRecv error: recv got EOF from 127.1.132.129:60757 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.403203 10715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.397274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:51809)
0923 22:13:04.397411 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.397424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.397501 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.397707 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.397708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.397905 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.397962 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.397964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.399032 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.399033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.400001 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.400005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.400513 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.400514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.400569 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.401045 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.401051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.403114 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
W20260923 22:13:04.403488 10714 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.397330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:51809 (local address 127.1.132.190:40917)
0923 22:13:04.397401 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.397404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.397479 (+ 75us) server_negotiation.cc:407] Connection header received
0923 22:13:04.397559 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.397560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.397589 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.397647 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.398053 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.398860 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.400111 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.400444 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.400513 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.403234 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:51809: BlockingRecv error: recv got EOF from 127.1.132.130:51809 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:04 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:04.525569 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:04.520593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:53489)
0923 22:13:04.520776 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:04.520789 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:04.520855 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:04.521107 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:04.521108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:04.521281 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:04.521327 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.521329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.521983 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.521984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.522494 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:04.522495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.522828 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.522830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:04.522882 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.523341 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:04.523346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:04.525489 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":76,"threads_started":1}
W20260923 22:13:04.525678 10714 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:04.520752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:53489 (local address 127.1.132.190:40917)
0923 22:13:04.520824 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:04.520826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:04.520838 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:13:04.521011 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:04.521013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:04.521030 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:04.521084 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:04.521404 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.521905 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.522578 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:04.522766 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:04.522822 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:04.525623 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:53489: BlockingRecv error: recv got EOF from 127.1.132.131:53489 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.265168 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.260282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:34061)
0923 22:13:05.260463 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.260477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.260543 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.260749 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.260750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.260945 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.260997 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.260998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.261741 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.261742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.262278 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.262279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.262664 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.262665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.262716 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.263204 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.263209 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.265099 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
W20260923 22:13:05.265314 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.260389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:34061 (local address 127.1.132.190:40917)
0923 22:13:05.260551 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.260553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.260562 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:05.260585 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.260586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.260629 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.260697 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.261114 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.261640 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.262371 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.262567 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.262635 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.265243 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:34061: BlockingRecv error: recv got EOF from 127.1.132.129:34061 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.411518 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.403726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:35161)
0923 22:13:05.403845 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.403857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.403926 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.404059 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.404060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.404287 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.404335 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.404337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.404970 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.404971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.405497 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.405498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.405866 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.405867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.405954 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.409544 (+ 3590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.409550 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.411455 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
W20260923 22:13:05.411644 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.403837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:35161 (local address 127.1.132.190:40917)
0923 22:13:05.403915 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.403917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.403926 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:05.403989 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.403990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.404002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.404045 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.404405 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.404900 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.405565 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.405776 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.405831 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.411582 (+ 5751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:35161: BlockingRecv error: recv got EOF from 127.1.132.130:35161 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:05 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:05.530838 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:05.526134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:59781)
0923 22:13:05.526295 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:05.526308 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:05.526377 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:05.526512 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:05.526513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:05.526708 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:05.526754 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.526756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.527438 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.527440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.527970 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:05.527972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.528350 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.528351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:05.528409 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.528841 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:05.528846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:05.530777 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
W20260923 22:13:05.530962 10718 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:05.526250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59781 (local address 127.1.132.190:40917)
0923 22:13:05.526314 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:05.526316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:05.526358 (+ 42us) server_negotiation.cc:407] Connection header received
0923 22:13:05.526437 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:05.526438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:05.526452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:05.526495 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:05.526833 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.527360 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.528049 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:05.528282 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:05.528333 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:05.530899 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:59781: BlockingRecv error: recv got EOF from 127.1.132.131:59781 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.270694 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.265918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:50299)
0923 22:13:06.266058 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.266071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.266141 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.266343 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.266344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.266537 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.266586 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.266588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.267244 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.267245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.267779 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.267781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.268116 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.268117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.268175 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.268665 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.268670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.270613 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
W20260923 22:13:06.270814 10722 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.266021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:50299 (local address 127.1.132.190:40917)
0923 22:13:06.266144 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.266145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.266154 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:06.266239 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.266240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.266251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.266315 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.266660 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.267165 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.267857 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.268052 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.268110 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.270749 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:50299: BlockingRecv error: recv got EOF from 127.1.132.129:50299 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.416546 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.412091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:48903)
0923 22:13:06.412259 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.412269 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.412328 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.412460 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.412461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.412628 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.412673 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.412675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.413321 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.413322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.413865 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.413866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.414199 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.414200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.414261 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.414712 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.414716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.416485 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":66,"threads_started":1}
W20260923 22:13:06.416666 10722 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.412263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:48903 (local address 127.1.132.190:40917)
0923 22:13:06.412332 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.412334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.412343 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:06.412389 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.412389 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.412401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.412445 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.412743 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.413239 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.413946 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.414136 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.414193 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.416587 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:48903: BlockingRecv error: recv got EOF from 127.1.132.130:48903 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:06 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:06.536419 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:06.531466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:52749)
0923 22:13:06.531601 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:06.531614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:06.531681 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:06.531874 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:06.531876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:06.532087 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:06.532143 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.532146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.532903 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.532904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.533438 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:06.533439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.533859 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.533860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:06.533911 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.534373 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:06.534378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:06.536358 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
W20260923 22:13:06.536552 10722 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:06.531540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:52749 (local address 127.1.132.190:40917)
0923 22:13:06.531593 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:06.531595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:06.531663 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:13:06.531777 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:06.531778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:06.531789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:06.531841 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:06.532250 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.532788 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.533548 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:06.533792 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:06.533845 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:06.536481 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:52749: BlockingRecv error: recv got EOF from 127.1.132.131:52749 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.276207 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.271239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:54747)
0923 22:13:07.271377 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.271389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.271454 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.271575 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.271575 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.271734 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.271782 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.271783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.272598 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.272599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.273164 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.273166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.273572 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.273573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.273625 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.274174 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.274179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.276106 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260923 22:13:07.276345 10726 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.271287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:54747 (local address 127.1.132.190:40917)
0923 22:13:07.271396 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.271397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.271438 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:13:07.271506 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.271506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.271516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.271561 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.271866 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.272505 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.273266 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.273479 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.273543 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.276257 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:54747: BlockingRecv error: recv got EOF from 127.1.132.129:54747 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.421772 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.417162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:41223)
0923 22:13:07.417331 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.417342 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.417411 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.417547 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.417548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.417727 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.417774 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.417776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.418428 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.418429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.418961 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.418963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.419299 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.419300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.419352 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.419876 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.419881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.421710 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
W20260923 22:13:07.421957 10726 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.417371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:41223 (local address 127.1.132.190:40917)
0923 22:13:07.417433 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.417435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.417444 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:07.417476 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.417477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.417487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.417533 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.417852 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.418349 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.419046 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.419237 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.419295 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.421900 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:41223: BlockingRecv error: recv got EOF from 127.1.132.130:41223 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:07 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:07.541559 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:07.536889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:48311)
0923 22:13:07.537008 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:07.537020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:07.537095 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:07.537218 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:07.537219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:07.537384 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:07.537427 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.537429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.538136 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.538136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.538672 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:07.538674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.539045 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.539046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:07.539102 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.539514 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:07.539518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:07.541477 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260923 22:13:07.541687 10726 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:07.536948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:48311 (local address 127.1.132.190:40917)
0923 22:13:07.537011 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:07.537013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:07.537082 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:13:07.537152 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:07.537153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:07.537166 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:07.537213 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:07.537492 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.538069 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.538746 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:07.538979 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:07.539029 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:07.541626 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:48311: BlockingRecv error: recv got EOF from 127.1.132.131:48311 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.281212 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.276686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:58231)
0923 22:13:08.276816 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.276827 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.276875 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.277054 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.277055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.277234 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.277285 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.277287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.277992 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.277993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.278541 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.278542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.278919 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.278920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.278958 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.279388 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.279392 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.281151 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260923 22:13:08.281315 10730 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.276781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58231 (local address 127.1.132.190:40917)
0923 22:13:08.276902 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.276903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.276913 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:08.276936 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.276936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.276945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.277008 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.277372 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.277895 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.278650 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.278852 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.278917 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.281242 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:58231: BlockingRecv error: recv got EOF from 127.1.132.129:58231 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.426839 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.422394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:33787)
0923 22:13:08.422542 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.422551 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.422599 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.422739 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.422739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.422915 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.422964 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.422966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.423630 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.423631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.424170 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.424173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.424594 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.424594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.424633 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.425027 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.425032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.426746 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
W20260923 22:13:08.426887 10730 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.422507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:33787 (local address 127.1.132.190:40917)
0923 22:13:08.422592 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.422594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.422606 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:13:08.422661 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.422662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.422672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.422733 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.423021 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.423528 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.424324 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.424529 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.424591 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.426834 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:33787: BlockingRecv error: recv got EOF from 127.1.132.130:33787 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:08 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:08.547077 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:08.542117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:57545)
0923 22:13:08.542269 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:08.542282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:08.542363 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:08.542500 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:08.542501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:08.542662 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:08.542709 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.542711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.543393 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.543394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.543990 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:08.543993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.544383 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.544384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:08.544443 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.544983 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:08.544988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:08.546985 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
W20260923 22:13:08.547158 10730 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:08.542220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:57545 (local address 127.1.132.190:40917)
0923 22:13:08.542291 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:08.542294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:08.542351 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:13:08.542424 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:08.542425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:08.542439 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:08.542489 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:08.542802 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.543321 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.544075 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:08.544321 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:08.544378 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:08.547100 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:57545: BlockingRecv error: recv got EOF from 127.1.132.131:57545 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.286525 10733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.281767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:35735)
0923 22:13:09.281897 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.281908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.281957 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.282200 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.282201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.282383 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.282436 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.282438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.283072 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.283073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.283628 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.283630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.283981 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.283982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.284033 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.284537 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.284542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.286436 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
W20260923 22:13:09.286630 10734 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.281923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:35735 (local address 127.1.132.190:40917)
0923 22:13:09.282032 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.282034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.282043 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:09.282069 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.282069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.282079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.282144 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.282503 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.283004 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.283700 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.283893 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.283944 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.286562 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:35735: BlockingRecv error: recv got EOF from 127.1.132.129:35735 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.432255 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.427429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:56727)
0923 22:13:09.427571 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.427584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.427657 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.427810 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.427811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.427976 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.428022 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.428024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.429006 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.429007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.429527 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.429529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.429901 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.429902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.429958 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.430411 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.430416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.432152 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
W20260923 22:13:09.432379 10734 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.427547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:56727 (local address 127.1.132.190:40917)
0923 22:13:09.427638 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.427640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.427654 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:13:09.427731 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.427732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.427744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.427792 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.428110 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.428935 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.429604 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.429835 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.429889 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.432308 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:56727: BlockingRecv error: recv got EOF from 127.1.132.130:56727 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:09 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:09.552119 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:09.547629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:33265)
0923 22:13:09.547756 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:09.547769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:09.547841 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:09.547990 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:09.547991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:09.548156 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:09.548227 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.548229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.548929 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.548930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.549613 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:09.549617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.549964 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.549965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:09.550018 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.550431 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:09.550436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:09.552059 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
W20260923 22:13:09.552271 10734 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:09.547735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:33265 (local address 127.1.132.190:40917)
0923 22:13:09.547785 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:09.547787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:09.547841 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:13:09.547887 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:09.547888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:09.547901 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:09.547947 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:09.548292 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.548804 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.549681 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:09.549881 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:09.549939 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:09.552156 (+ 2217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:33265: BlockingRecv error: recv got EOF from 127.1.132.131:33265 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.291610 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.287098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:51729)
0923 22:13:10.287224 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.287237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.287287 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.287458 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.287459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.287639 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.287694 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.287696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.288384 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.288385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.288948 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.288950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.289307 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.289308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.289352 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.289814 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.289818 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.291536 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
W20260923 22:13:10.291785 10738 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.287190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51729 (local address 127.1.132.190:40917)
0923 22:13:10.287312 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.287314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.287321 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:10.287344 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.287344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.287354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.287418 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.287760 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.288310 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.289037 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.289241 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.289295 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.291714 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:51729: BlockingRecv error: recv got EOF from 127.1.132.129:51729 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.441406 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.432741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:46723)
0923 22:13:10.432928 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.432939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.433007 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.433128 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.433128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.433286 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.433334 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.433336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.434016 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.434016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.434580 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.434582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.434979 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.434980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.435038 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.439315 (+ 4277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.439321 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.441335 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
W20260923 22:13:10.441512 10738 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.432832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:46723 (local address 127.1.132.190:40917)
0923 22:13:10.432881 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.432884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.432990 (+ 106us) server_negotiation.cc:407] Connection header received
0923 22:13:10.433065 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.433066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.433077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.433114 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.433390 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.433949 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.434645 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.434900 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.434968 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.441452 (+ 6484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:46723: BlockingRecv error: recv got EOF from 127.1.132.130:46723 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:10 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:10.557253 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:10.552633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:57887)
0923 22:13:10.552790 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:10.552802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:10.552883 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:10.553018 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:10.553019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:10.553193 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:10.553240 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.553242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.553881 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.553882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.554401 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:10.554403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.554714 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.554714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:10.554768 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.555207 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:10.555212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:10.557181 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
W20260923 22:13:10.557337 10738 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:10.552688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:57887 (local address 127.1.132.190:40917)
0923 22:13:10.552779 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:10.552782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:10.552874 (+ 92us) server_negotiation.cc:407] Connection header received
0923 22:13:10.552947 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:10.552948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:10.552961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:10.553007 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:10.553309 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.553806 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.554473 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:10.554655 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:10.554709 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:10.557280 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:57887: BlockingRecv error: recv got EOF from 127.1.132.131:57887 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.297191 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.292074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:34069)
0923 22:13:11.292257 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.292273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.292351 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.292551 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.292553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.292808 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.292872 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.292875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.293622 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.293623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.294209 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.294211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.294672 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.294673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.294729 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.295227 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.295232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.297095 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":56,"threads_started":1}
W20260923 22:13:11.297320 10742 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.292230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:34069 (local address 127.1.132.190:40917)
0923 22:13:11.292392 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.292394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.292404 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:11.292428 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.292429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.292439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.292506 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.292999 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.293519 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.294321 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.294525 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.294590 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.297254 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:34069: BlockingRecv error: recv got EOF from 127.1.132.129:34069 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.446172 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.441863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:46223)
0923 22:13:11.442008 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.442019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.442080 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.442191 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.442191 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.442351 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.442398 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.442400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.443056 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.443056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.443586 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.443588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.443943 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.443944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.443981 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.444411 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.444416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.446107 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260923 22:13:11.446281 10742 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.441951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:46223 (local address 127.1.132.190:40917)
0923 22:13:11.442028 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.442029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.442078 (+ 49us) server_negotiation.cc:407] Connection header received
0923 22:13:11.442124 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.442125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.442135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.442185 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.442488 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.442989 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.443679 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.443876 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.443949 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.446219 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:46223: BlockingRecv error: recv got EOF from 127.1.132.130:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:11 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:11.562052 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:11.557726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:59261)
0923 22:13:11.557889 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:11.557899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:11.557950 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:11.558072 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:11.558072 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:11.558237 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:11.558294 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.558296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.558888 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.558888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.559413 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:11.559415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.559755 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.559756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:11.559796 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.560215 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:11.560219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:11.561994 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
W20260923 22:13:11.562136 10742 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:11.557865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:59261 (local address 127.1.132.190:40917)
0923 22:13:11.557944 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:11.557946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:11.557957 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:13:11.558004 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:11.558005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:11.558016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:11.558056 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:11.558351 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.558832 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.559498 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:11.559698 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:11.559753 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:11.562079 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:59261: BlockingRecv error: recv got EOF from 127.1.132.131:59261 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.302752 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.297786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:34643)
0923 22:13:12.297920 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.297932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.297982 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.298113 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.298114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.298306 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.298353 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.298355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.299059 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.299059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.299689 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.299693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.300093 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.300095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.300149 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.300679 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.300683 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.302671 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260923 22:13:12.302881 10746 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.297841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:34643 (local address 127.1.132.190:40917)
0923 22:13:12.297957 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.297959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.297969 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:12.298040 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.298041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.298050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.298098 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.298420 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.298955 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.299792 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.300001 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.300056 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.302803 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:34643: BlockingRecv error: recv got EOF from 127.1.132.129:34643 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":54,"threads_started":1}
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.451365 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.446734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:41117)
0923 22:13:12.446890 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.446903 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.446968 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.447133 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.447134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.447300 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.447342 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.447344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.448043 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.448044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.448646 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.448649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.448993 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.448994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.449047 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.449494 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.449499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.451288 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
W20260923 22:13:12.451454 10746 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.446868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:41117 (local address 127.1.132.190:40917)
0923 22:13:12.446947 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.446949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.446959 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:12.447041 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.447042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.447053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.447112 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.447418 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.447961 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.448733 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.448931 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.448985 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.451395 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:41117: BlockingRecv error: recv got EOF from 127.1.132.130:41117 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:12 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:12.566975 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:12.562601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:46825)
0923 22:13:12.562788 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:12.562802 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:12.562880 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:12.563024 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:12.563025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:12.563193 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:12.563237 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.563239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.563939 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.563940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.564509 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:12.564511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.564828 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.564830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:12.564881 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.565310 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:12.565314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:12.566908 (+ 1594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":67,"threads_started":1}
W20260923 22:13:12.567137 10746 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:12.562752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:46825 (local address 127.1.132.190:40917)
0923 22:13:12.562826 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:12.562829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:12.562861 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:13:12.562943 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:12.562944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:12.562959 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:12.563003 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:12.563319 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.563876 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.564589 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:12.564771 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:12.564824 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:12.567043 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:46825: BlockingRecv error: recv got EOF from 127.1.132.131:46825 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:13.308044 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.303239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:53315)
0923 22:13:13.303396 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.303413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.303477 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.303592 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.303592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.303761 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.303807 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.303808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.304648 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.304649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.305197 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.305199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.305599 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.305600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.305658 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.306138 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:13.306143 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.307980 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
W20260923 22:13:13.308231 10750 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.303326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:53315 (local address 127.1.132.190:40917)
0923 22:13:13.303431 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.303432 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.303460 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:13:13.303521 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.303522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.303531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.303621 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.303895 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.304525 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.305288 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.305496 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.305562 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.308134 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:53315: BlockingRecv error: recv got EOF from 127.1.132.129:53315 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":49,"threads_started":1}
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:13.456498 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.451908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:34389)
0923 22:13:13.452072 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.452083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.452153 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.452310 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.452311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.452493 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.452549 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.452551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.453385 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.453386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.454025 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.454028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.454368 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.454369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.454405 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.454791 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:13.454797 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.456428 (+ 1631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
W20260923 22:13:13.456602 10750 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.452027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:34389 (local address 127.1.132.190:40917)
0923 22:13:13.452078 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.452080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.452149 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:13:13.452232 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.452233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.452244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.452295 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.452646 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.453319 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.454101 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.454300 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.454366 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.456540 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:34389: BlockingRecv error: recv got EOF from 127.1.132.130:34389 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:13 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:13.571928 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:13.567476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:49373)
0923 22:13:13.567610 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:13.567623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:13.567691 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:13.567843 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:13.567843 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:13.568009 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:13.568051 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.568052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.568704 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.568705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.569269 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:13.569271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.569712 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.569713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:13.569764 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.570198 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:13.570202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:13.571859 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260923 22:13:13.572005 10750 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:13.567584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:49373 (local address 127.1.132.190:40917)
0923 22:13:13.567639 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:13.567640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:13.567674 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:13:13.567751 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:13.567752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:13.567763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:13.567809 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:13.568116 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.568637 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.569371 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:13.569580 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:13.569678 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:13.571948 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:49373: BlockingRecv error: recv got EOF from 127.1.132.131:49373 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:14.313032 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.308551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:38345)
0923 22:13:14.308670 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.308681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.308755 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.308885 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.308885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.309050 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.309095 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.309096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.309755 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.309756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.310291 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.310293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.310620 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.310621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.310655 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.311096 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:14.311100 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.312961 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
W20260923 22:13:14.313149 10754 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:14.308612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:38345 (local address 127.1.132.190:40917)
0923 22:13:14.308753 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.308755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.308765 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:14.308795 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.308796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.308807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.308874 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.309155 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.309673 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.310371 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.310564 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.310616 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.313078 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:38345: BlockingRecv error: recv got EOF from 127.1.132.129:38345 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:14.464270 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.457104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:53043)
0923 22:13:14.457231 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.457243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.457307 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.457466 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.457467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.457642 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.457725 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.457727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.458576 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.458577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.460132 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.460138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.460548 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.460549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.460604 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.461087 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:14.461093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.464162 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260923 22:13:14.464382 10754 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:14.457244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:53043 (local address 127.1.132.190:40917)
0923 22:13:14.457289 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.457291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.457300 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:14.457371 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.457372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.457405 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.457448 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.457796 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.458471 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.460241 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.460480 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.460540 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.464320 (+ 3780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:53043: BlockingRecv error: recv got EOF from 127.1.132.130:53043 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:14 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:14.577553 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:14.572406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:58281)
0923 22:13:14.572526 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:14.572536 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:14.572595 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:14.572744 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:14.572745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:14.573038 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:14.573095 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.573097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.573830 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.573831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.574447 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:14.574450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.574862 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.574863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:14.574910 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.575331 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:14.575336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:14.577493 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
W20260923 22:13:14.577680 10754 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:14.572468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:58281 (local address 127.1.132.190:40917)
0923 22:13:14.572522 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:14.572524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:14.572644 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:13:14.572669 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:14.572670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:14.572684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:14.572734 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:14.573205 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.573740 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.574538 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:14.574783 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:14.574847 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:14.577612 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:58281: BlockingRecv error: recv got EOF from 127.1.132.131:58281 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:15.318079 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.313572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:51351)
0923 22:13:15.313709 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:15.313719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:15.313771 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:15.313951 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:15.313952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:15.314118 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:15.314166 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.314167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.314878 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.314879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.315435 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.315437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.315797 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.315798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.315843 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.316353 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:15.316358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:15.318019 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
W20260923 22:13:15.318222 10758 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:15.313650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51351 (local address 127.1.132.190:40917)
0923 22:13:15.313828 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.313829 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.313838 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:13:15.313861 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.313862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.313872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.313936 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.314277 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.314795 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.315525 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.315729 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.315786 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.318142 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:51351: BlockingRecv error: recv got EOF from 127.1.132.129:51351 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:15.475778 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.465053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:38429)
0923 22:13:15.465215 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:15.465227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:15.465331 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:15.465636 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:15.465639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:15.465878 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:15.465948 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.465951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.466793 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.466795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.467362 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.467365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.467741 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.467742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.467799 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.473522 (+ 5723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:15.473529 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:15.475665 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":63,"threads_started":1}
W20260923 22:13:15.475873 10758 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:15.465163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:38429 (local address 127.1.132.190:40917)
0923 22:13:15.465231 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.465232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.465292 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:13:15.465394 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.465395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.465408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.465615 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.466050 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.466618 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.467454 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.467671 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.467727 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.475805 (+ 8078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:38429: BlockingRecv error: recv got EOF from 127.1.132.130:38429 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:15 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.131@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:15.584148 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:15.578248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.131:37041)
0923 22:13:15.578411 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:15.578424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:15.578490 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:15.578662 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:15.578664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:15.578833 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:15.578885 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.578886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.579693 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.579694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.580375 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:15.580379 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.580771 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.580772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:15.580823 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.581688 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:15.581693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:15.584054 (+ 2361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":60,"threads_started":1}
W20260923 22:13:15.584290 10758 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:15.578383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:37041 (local address 127.1.132.190:40917)
0923 22:13:15.578471 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:15.578473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:15.578483 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:15.578547 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:15.578548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:15.578561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:15.578603 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:15.579036 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.579562 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.580480 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:15.580678 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:15.580741 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:15.584224 (+ 3483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.131:37041: BlockingRecv error: recv got EOF from 127.1.132.131:37041 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 23 22:13:16 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:16 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.129@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:16.322963 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.318587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.129:50179)
0923 22:13:16.318724 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.318735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.318786 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.318913 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.318913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.319070 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:16.319121 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.319122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.319755 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.319756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.320325 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.320327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.320636 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.320636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.320672 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.321085 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:16.321089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:16.322893 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260923 22:13:16.323199 10762 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:16.318675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:50179 (local address 127.1.132.190:40917)
0923 22:13:16.318773 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.318775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.318785 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:16.318838 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.318838 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.318849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.318903 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.319178 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.319693 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.320391 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.320582 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.320631 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.323131 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.129:50179: BlockingRecv error: recv got EOF from 127.1.132.129:50179 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":46,"threads_started":1}
Sep 23 22:13:16 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:13:16 dist-test-slave-l4q5 krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.132.130@KRBTEST.COM for oryx/127.1.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:13:16.481230 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.476369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40917 (local address 127.1.132.130:42601)
0923 22:13:16.476480 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.476493 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.476569 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.476712 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.476713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.476878 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:16.476926 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.476928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.477572 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.477573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.478114 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.478117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.478464 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.478465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.478520 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.478977 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:16.478981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:16.481109 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.132.190:40917: Server oryx/127.1.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
W20260923 22:13:16.481298 10762 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:16.476457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:42601 (local address 127.1.132.190:40917)
0923 22:13:16.476529 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.476531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.476547 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:13:16.476637 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.476638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.476649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.476698 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.476999 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.477494 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.478205 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.478398 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.478455 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.481241 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.132.130:42601: BlockingRecv error: recv got EOF from 127.1.132.130:42601 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:13:16.503199 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10288
I20260923 22:13:16.507308 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10439
I20260923 22:13:16.511080 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10571
I20260923 22:13:16.514827 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10217
2026-09-23T22:13:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17418 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T22:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:16Z Disabled control of system clock
I20260923 22:13:16.540683 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:43341
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:43341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:13:16.621515 10767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:16.621860 10767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:16.621883 10767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:16.623138 10767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:16.623176 10767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:16.623193 10767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:16.623207 10767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:16.623219 10767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:16.624632 10767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:43341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:43341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:16.624861 10767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:16.625121 10767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:16.630074 10772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.630069 10773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:16.630095 10767 server_base.cc:1034] running on GCE node
W20260923 22:13:16.630066 10775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:16.630679 10767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:16.631006 10767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:16.632146 10767 hybrid_clock.cc:648] HybridClock initialized: now 1790201596632134 us; error 34 us; skew 500 ppm
I20260923 22:13:16.633615 10767 webserver.cc:460] Webserver started at http://127.1.132.190:46565/ using document root <none> and password file <none>
I20260923 22:13:16.633908 10767 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:16.634240 10767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:16.634408 10767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:16.635929 10767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "9e31b3eec4a94bbeb032966c073c1c6a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "62cdb6f0f7f96122bc336f840054b2ec"
server_key_iv: "6498b8d0cf89f97c3232c43627411708"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.636647 10767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "9e31b3eec4a94bbeb032966c073c1c6a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "62cdb6f0f7f96122bc336f840054b2ec"
server_key_iv: "6498b8d0cf89f97c3232c43627411708"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.638247 10767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:13:16.639163 10781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.639384 10767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:16.639473 10767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "9e31b3eec4a94bbeb032966c073c1c6a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "62cdb6f0f7f96122bc336f840054b2ec"
server_key_iv: "6498b8d0cf89f97c3232c43627411708"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.639546 10767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:16.664877 10767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:16.665207 10767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:16.665341 10767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:16.669016 10767 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:43341
I20260923 22:13:16.669101 10833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:43341 every 8 connection(s)
I20260923 22:13:16.669394 10767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:16.670143 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:16.672834 10834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a: Bootstrap starting.
I20260923 22:13:16.673378 10834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:16.673616 10834 log.cc:824] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:16.674292 10834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a: No bootstrap required, opened a new log
I20260923 22:13:16.675387 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10767
I20260923 22:13:16.675494 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:13:16.675676 1554 external_mini_cluster.cc:1428] Setting key 48e79cdaddd34b08961945ae2a7e98c6
I20260923 22:13:16.676246 10834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } }
I20260923 22:13:16.676438 10834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:16.676467 10834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e31b3eec4a94bbeb032966c073c1c6a, State: Initialized, Role: FOLLOWER
I20260923 22:13:16.676600 10834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } }
I20260923 22:13:16.676671 10834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:16.676692 10834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:16.676736 10834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:16.677402 10834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } }
I20260923 22:13:16.677492 10834 leader_election.cc:302] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e31b3eec4a94bbeb032966c073c1c6a; no voters:
I20260923 22:13:16.677662 10834 leader_election.cc:288] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:16.677702 10839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:16.677809 10839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [term 1 LEADER]: Becoming Leader. State: Replica: 9e31b3eec4a94bbeb032966c073c1c6a, State: Running, Role: LEADER
I20260923 22:13:16.677893 10839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } }
I20260923 22:13:16.677937 10834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:16.678399 10840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } } }
I20260923 22:13:16.678401 10839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e31b3eec4a94bbeb032966c073c1c6a. Latest consensus state: current_term: 1 leader_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e31b3eec4a94bbeb032966c073c1c6a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43341 } } }
I20260923 22:13:16.678506 10840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:16.678510 10839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:16.678787 10843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:16.679306 10843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:16.681066 10843 catalog_manager.cc:1347] Generated new cluster ID: f19f5b4ee6634527af5e80e6bd7a8ec4
I20260923 22:13:16.681134 10843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:16.682351 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.676266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.1.132.190:43341)
0923 22:13:16.676479 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.676488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.676521 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:13:16.676644 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.676650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.676733 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.676872 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.677061 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.679039 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.680337 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.680927 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.681054 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.681744 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:16.681747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:16.681748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:16.681911 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:16.681944 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:16.681955 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:16.682006 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.682067 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.682139 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260923 22:13:16.696619 10843 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:16.697132 10843 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:16.701884 10843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e31b3eec4a94bbeb032966c073c1c6a: Generated new TSK 0
I20260923 22:13:16.702072 10843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:13:16.707789 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--builtin_ntp_servers=127.1.132.148:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:13:16.796145 10858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:16.796386 10858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:16.796407 10858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:16.797674 10858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:16.797734 10858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:16.797760 10858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:13:16.799206 10858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:16.799481 10858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:16.799964 10858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:16.805903 10863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.805912 10866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.805989 10864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:16.806321 10858 server_base.cc:1034] running on GCE node
I20260923 22:13:16.806527 10858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:16.806777 10858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:16.808001 10858 hybrid_clock.cc:648] HybridClock initialized: now 1790201596807983 us; error 31 us; skew 500 ppm
I20260923 22:13:16.809873 10858 webserver.cc:460] Webserver started at http://127.1.132.129:44565/ using document root <none> and password file <none>
I20260923 22:13:16.810123 10858 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:16.810176 10858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:16.810312 10858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:16.811403 10858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "a28f698a931d4252ac9e7316e4fd8e9a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "99c82b10bbe3bc9ad380fdbbe1ddcbd3"
server_key_iv: "a301e63902c5d3e732416f1072a390c9"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.811758 10858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "a28f698a931d4252ac9e7316e4fd8e9a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "99c82b10bbe3bc9ad380fdbbe1ddcbd3"
server_key_iv: "a301e63902c5d3e732416f1072a390c9"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.813143 10858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:16.814397 10872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.814832 10858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:16.815037 10858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "a28f698a931d4252ac9e7316e4fd8e9a"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "99c82b10bbe3bc9ad380fdbbe1ddcbd3"
server_key_iv: "a301e63902c5d3e732416f1072a390c9"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.815281 10858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:16.834420 10858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:16.834703 10858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:16.834812 10858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:16.835006 10858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:16.835274 10858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:16.835311 10858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.835333 10858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:16.835345 10858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.840876 10858 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:41843
I20260923 22:13:16.840922 10985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:41843 every 8 connection(s)
I20260923 22:13:16.841252 10858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:16.843349 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10858
I20260923 22:13:16.843448 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:16.843641 1554 external_mini_cluster.cc:1428] Setting key b3e2013a91c996b0f9aad791cbf7e1f9
I20260923 22:13:16.844633 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--builtin_ntp_servers=127.1.132.148:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 22:13:16.847030 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.841956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:47311 (local address 127.1.132.190:43341)
0923 22:13:16.842015 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.842017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.842245 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:13:16.842832 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.842833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.842849 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.842977 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.843881 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.844494 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.845512 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.845704 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.845759 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.846715 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:16.846716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:16.846716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:16.846724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:16.846738 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:16.846742 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:16.846804 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.846923 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.846966 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:13:16.847227 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.841938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43341 (local address 127.1.132.129:47311)
0923 22:13:16.842111 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.842146 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.842732 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.843074 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.843078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.843087 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:16.843787 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.843795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.844576 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.844577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.845429 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.845432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.845771 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.845772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.845927 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.846520 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:16.846520 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:16.846563 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:16.846593 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:16.846817 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:16.846824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:16.846827 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.846897 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.847008 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260923 22:13:16.847679 10986 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43341
I20260923 22:13:16.847821 10986 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:16.848001 10986 heartbeater.cc:511] Master 127.1.132.190:43341 requested a full tablet report, sending...
I20260923 22:13:16.848573 10798 ts_manager.cc:194] Registered new tserver with Master: a28f698a931d4252ac9e7316e4fd8e9a (127.1.132.129:41843)
I20260923 22:13:16.849472 10798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.129:47311
W20260923 22:13:16.926079 10989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:16.926303 10989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:16.926324 10989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:16.927441 10989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:16.927475 10989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:16.927500 10989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:13:16.928828 10989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:16.929028 10989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:16.929284 10989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:16.933997 10997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.934026 10994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:16.934012 10995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:16.934274 10989 server_base.cc:1034] running on GCE node
I20260923 22:13:16.934489 10989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:16.934705 10989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:16.935860 10989 hybrid_clock.cc:648] HybridClock initialized: now 1790201596935844 us; error 26 us; skew 500 ppm
I20260923 22:13:16.937309 10989 webserver.cc:460] Webserver started at http://127.1.132.130:45807/ using document root <none> and password file <none>
I20260923 22:13:16.937547 10989 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:16.937597 10989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:16.937723 10989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:16.938755 10989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "46f5ba639cf2427185b3113b91eae15d"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "bdced843dd7c1d887e9755f67876f98f"
server_key_iv: "aa489448f0278b22fd5e11c1bcc95252"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.939083 10989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "46f5ba639cf2427185b3113b91eae15d"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "bdced843dd7c1d887e9755f67876f98f"
server_key_iv: "aa489448f0278b22fd5e11c1bcc95252"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.940356 10989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:16.941085 11003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.941278 10989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:16.941359 10989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "46f5ba639cf2427185b3113b91eae15d"
format_stamp: "Formatted at 2026-09-23 22:13:16 on dist-test-slave-l4q5"
server_key: "bdced843dd7c1d887e9755f67876f98f"
server_key_iv: "aa489448f0278b22fd5e11c1bcc95252"
server_key_version: "encryptionkey@0"
I20260923 22:13:16.941431 10989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:16.954119 10989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:16.954434 10989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:16.954566 10989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:16.954785 10989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:16.955114 10989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:16.955147 10989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.955175 10989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:16.955192 10989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:16.960618 10989 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39741
I20260923 22:13:16.960681 11116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39741 every 8 connection(s)
I20260923 22:13:16.960973 10989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:16.966527 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.961828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:45141 (local address 127.1.132.190:43341)
0923 22:13:16.961887 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:16.961889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:16.962103 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:13:16.962696 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:16.962698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:16.962716 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:16.962772 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:16.963480 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.964015 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.965117 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.965318 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.965378 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.966288 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:16.966289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:16.966290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:16.966295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:16.966309 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:16.966313 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:16.966358 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:16.966438 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:13:16.966464 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:13:16.966769 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:16.961764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43341 (local address 127.1.132.130:45141)
0923 22:13:16.961982 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:16.962018 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:16.962573 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:16.962829 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:16.962833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:16.962843 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:16.963383 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.963388 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.964125 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.964125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.965004 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:16.965008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:16.965429 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:16.965430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:16.965561 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:16.966145 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:16.966145 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:16.966175 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:16.966199 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:16.966377 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:16.966380 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:16.966382 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:13:16.966432 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:13:16.966521 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260923 22:13:16.967330 11117 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43341
I20260923 22:13:16.967424 11117 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:16.967602 11117 heartbeater.cc:511] Master 127.1.132.190:43341 requested a full tablet report, sending...
I20260923 22:13:16.967973 10798 ts_manager.cc:194] Registered new tserver with Master: 46f5ba639cf2427185b3113b91eae15d (127.1.132.130:39741)
I20260923 22:13:16.968358 10798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.130:45141
I20260923 22:13:16.970077 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 10989
I20260923 22:13:16.970155 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:16.970363 1554 external_mini_cluster.cc:1428] Setting key 97e4f269f75637a254bd7fdc525cd3a5
I20260923 22:13:16.971292 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--builtin_ntp_servers=127.1.132.148:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:13:17.052726 11120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:17.052948 11120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:17.052970 11120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:17.054147 11120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:17.054189 11120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:17.054214 11120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:13:17.055613 11120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:17.055838 11120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:17.056107 11120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:17.061157 11128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.061169 11126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.061177 11125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.061492 11120 server_base.cc:1034] running on GCE node
I20260923 22:13:17.061733 11120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:17.062036 11120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:17.063414 11120 hybrid_clock.cc:648] HybridClock initialized: now 1790201597063363 us; error 60 us; skew 500 ppm
I20260923 22:13:17.064896 11120 webserver.cc:460] Webserver started at http://127.1.132.131:43015/ using document root <none> and password file <none>
I20260923 22:13:17.065156 11120 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:17.065212 11120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:17.065344 11120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:17.066427 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "60e687bb1d504f9d8818459b9f26a47e"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "66526b5fa3c727a8406fbfb7a42e901c"
server_key_iv: "79cc3da82a14fddd710e335197d937e3"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.066773 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "60e687bb1d504f9d8818459b9f26a47e"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "66526b5fa3c727a8406fbfb7a42e901c"
server_key_iv: "79cc3da82a14fddd710e335197d937e3"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.068089 11120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:13:17.068977 11134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.069200 11120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:17.069295 11120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "60e687bb1d504f9d8818459b9f26a47e"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "66526b5fa3c727a8406fbfb7a42e901c"
server_key_iv: "79cc3da82a14fddd710e335197d937e3"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.069366 11120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:17.081668 11120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:17.081972 11120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:17.082098 11120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:17.082316 11120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:17.082628 11120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:17.082659 11120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.082700 11120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:17.082720 11120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.088414 11120 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:33091
I20260923 22:13:17.088464 11247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:33091 every 8 connection(s)
I20260923 22:13:17.088799 11120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:17.094060 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.089583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:53065 (local address 127.1.132.190:43341)
0923 22:13:17.089667 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.089669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.089874 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:13:17.090390 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.090391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.090404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.090456 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.091140 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:17.091648 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:17.092696 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:17.092902 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:17.092958 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:17.093767 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:17.093768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:17.093769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:17.093774 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:17.093790 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:17.093794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:17.093839 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:17.093921 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:13:17.093974 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 22:13:17.094244 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.089524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43341 (local address 127.1.132.131:53065)
0923 22:13:17.089748 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:17.089792 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:17.090294 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:17.090525 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:17.090528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:17.090537 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:17.091048 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:17.091053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:17.091737 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:17.091738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:17.092604 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:17.092606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:17.092975 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:17.092976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:17.093105 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:17.093624 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:17.093624 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:17.093654 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:17.093678 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:17.093847 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:17.093852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:17.093856 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:17.093917 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:13:17.094024 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260923 22:13:17.094803 11248 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43341
I20260923 22:13:17.094888 11248 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:17.095155 11248 heartbeater.cc:511] Master 127.1.132.190:43341 requested a full tablet report, sending...
I20260923 22:13:17.095623 10798 ts_manager.cc:194] Registered new tserver with Master: 60e687bb1d504f9d8818459b9f26a47e (127.1.132.131:33091)
I20260923 22:13:17.095976 10798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.131:53065
I20260923 22:13:17.096163 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11120
I20260923 22:13:17.096284 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:17.096486 1554 external_mini_cluster.cc:1428] Setting key 4c78417589ed0d826a45959d8e04ba36
I20260923 22:13:17.097321 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:13:17.098987 10838 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:17.098586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.1.132.190:43341)
0923 22:13:17.098627 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.098629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.098715 (+ 86us) server_negotiation.cc:407] Connection header received
0923 22:13:17.098783 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.098784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.098796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.098834 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.098940 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36674: BlockingRecv error: recv got EOF from 127.0.0.1:36674 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:13:17.099041 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10858
I20260923 22:13:17.103519 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10989
I20260923 22:13:17.107496 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11120
I20260923 22:13:17.111011 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 10767
2026-09-23T22:13:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (595 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T22:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:17Z Disabled control of system clock
I20260923 22:13:17.135617 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35709
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:17.220425 11259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:17.220638 11259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:17.220659 11259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:17.221850 11259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:17.221891 11259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:17.221903 11259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:17.221915 11259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:17.221925 11259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:17.223255 11259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:17.223479 11259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:17.223747 11259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:17.228521 11265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.228533 11264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.228562 11259 server_base.cc:1034] running on GCE node
W20260923 22:13:17.228528 11267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.228971 11259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:17.229240 11259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:17.230414 11259 hybrid_clock.cc:648] HybridClock initialized: now 1790201597230375 us; error 51 us; skew 500 ppm
I20260923 22:13:17.231709 11259 webserver.cc:460] Webserver started at http://127.1.132.190:38453/ using document root <none> and password file <none>
I20260923 22:13:17.231940 11259 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:17.231983 11259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:17.232101 11259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:17.233160 11259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "6b23775458dc4c62a5960f18eb3506d6"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "4368c6c321d9269d98bb25a538a25263"
server_key_iv: "1f0d9c061889126120eb7d7be5d313ea"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.233477 11259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "6b23775458dc4c62a5960f18eb3506d6"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "4368c6c321d9269d98bb25a538a25263"
server_key_iv: "1f0d9c061889126120eb7d7be5d313ea"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.234731 11259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:13:17.235483 11273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.235661 11259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:17.235738 11259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "6b23775458dc4c62a5960f18eb3506d6"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "4368c6c321d9269d98bb25a538a25263"
server_key_iv: "1f0d9c061889126120eb7d7be5d313ea"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.235816 11259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:17.239244 11259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:17.239504 11259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:17.239638 11259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:17.243312 11259 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35709
I20260923 22:13:17.243342 11325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35709 every 8 connection(s)
I20260923 22:13:17.243652 11259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:17.244333 11326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:17.246989 11326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6: Bootstrap starting.
I20260923 22:13:17.247545 11326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:17.247816 11326 log.cc:824] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:17.248517 11326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6: No bootstrap required, opened a new log
I20260923 22:13:17.250115 11326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } }
I20260923 22:13:17.250300 11326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:17.250330 11326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b23775458dc4c62a5960f18eb3506d6, State: Initialized, Role: FOLLOWER
I20260923 22:13:17.250483 11326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } }
I20260923 22:13:17.250504 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11259
I20260923 22:13:17.250563 11326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:17.250594 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:13:17.250598 11326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:17.250641 11326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:17.250797 1554 external_mini_cluster.cc:1428] Setting key 6942ece90bf30cb7b2910f8f12887849
I20260923 22:13:17.251320 11326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } }
I20260923 22:13:17.251432 11326 leader_election.cc:302] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b23775458dc4c62a5960f18eb3506d6; no voters:
I20260923 22:13:17.251603 11326 leader_election.cc:288] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:17.251657 11331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:17.251793 11331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6b23775458dc4c62a5960f18eb3506d6, State: Running, Role: LEADER
I20260923 22:13:17.251863 11326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:17.251879 11331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } }
I20260923 22:13:17.252702 11333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b23775458dc4c62a5960f18eb3506d6. Latest consensus state: current_term: 1 leader_uuid: "6b23775458dc4c62a5960f18eb3506d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } } }
I20260923 22:13:17.252796 11333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:17.252991 11339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:17.253546 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b23775458dc4c62a5960f18eb3506d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b23775458dc4c62a5960f18eb3506d6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35709 } } }
I20260923 22:13:17.253639 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:17.253835 11339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:17.253867 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.251395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.1.132.190:35709)
0923 22:13:17.251673 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.251682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.251722 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:13:17.251850 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.251858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.251907 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.252343 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.253252 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:17.253256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:17.253256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:17.253260 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:17.253310 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:17.253327 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:17.253407 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:17.253492 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:13:17.253554 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":78,"threads_started":1}
I20260923 22:13:17.255371 11339 catalog_manager.cc:1347] Generated new cluster ID: a15863b040ee4d88928bfc5046452118
I20260923 22:13:17.255426 11339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:13:17.271194 11339 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:17.271404 11339 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:17.277383 11339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b23775458dc4c62a5960f18eb3506d6: Generated new TSK 0
I20260923 22:13:17.277649 11339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:13:17.284149 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--builtin_ntp_servers=127.1.132.148:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:17.368073 11350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:17.368320 11350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:17.368355 11350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:17.369649 11350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:17.369696 11350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:17.369742 11350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:13:17.371182 11350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:17.371435 11350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:17.371732 11350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:17.377552 11358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.377662 11356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.377552 11355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.378011 11350 server_base.cc:1034] running on GCE node
I20260923 22:13:17.378206 11350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:17.378453 11350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:17.379613 11350 hybrid_clock.cc:648] HybridClock initialized: now 1790201597379598 us; error 29 us; skew 500 ppm
I20260923 22:13:17.380867 11350 webserver.cc:460] Webserver started at http://127.1.132.129:34039/ using document root <none> and password file <none>
I20260923 22:13:17.381091 11350 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:17.381139 11350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:17.381291 11350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:17.382372 11350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "d02ebe09435244a79e826345a567613c"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "c192419a21190b685317b0a10173a737"
server_key_iv: "fb9df3f9c67bae31cbaea32e25e09a8a"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.382699 11350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "d02ebe09435244a79e826345a567613c"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "c192419a21190b685317b0a10173a737"
server_key_iv: "fb9df3f9c67bae31cbaea32e25e09a8a"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.384004 11350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:17.384768 11364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.384907 11350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.384984 11350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "d02ebe09435244a79e826345a567613c"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "c192419a21190b685317b0a10173a737"
server_key_iv: "fb9df3f9c67bae31cbaea32e25e09a8a"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.385051 11350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:17.388286 11350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:17.388531 11350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:17.388671 11350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:17.388854 11350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:17.389154 11350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:17.389201 11350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.389228 11350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:17.389250 11350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.394585 11350 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:33437
I20260923 22:13:17.394634 11477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:33437 every 8 connection(s)
I20260923 22:13:17.394928 11350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:13:17.397831 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.395590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:35841 (local address 127.1.132.190:35709)
0923 22:13:17.395640 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.395642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.395843 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:13:17.396414 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.396416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.396425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.396475 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.397591 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:17.397593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:17.397593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:17.397598 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:17.397616 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:17.397621 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:17.397671 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:17.397742 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:13:17.397760 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:13:17.398013 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.395568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35709 (local address 127.1.132.129:35841)
0923 22:13:17.395731 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:17.395765 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:17.396318 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:17.396543 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:17.396547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:17.396557 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:17.397441 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:17.397441 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:17.397470 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:17.397498 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:17.397675 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:17.397680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:17.397683 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:17.397754 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:13:17.397819 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260923 22:13:17.398452 11478 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35709
I20260923 22:13:17.398541 11478 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:17.398684 11478 heartbeater.cc:511] Master 127.1.132.190:35709 requested a full tablet report, sending...
I20260923 22:13:17.399207 11290 ts_manager.cc:194] Registered new tserver with Master: d02ebe09435244a79e826345a567613c (127.1.132.129:33437)
I20260923 22:13:17.399273 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11350
I20260923 22:13:17.399356 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:17.399564 1554 external_mini_cluster.cc:1428] Setting key ebb86bb00b332142793d9a8b2b598d1d
I20260923 22:13:17.400507 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--builtin_ntp_servers=127.1.132.148:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:17.482806 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:17.483021 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:17.483054 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:17.484267 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:17.484308 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:17.484342 11481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:13:17.485678 11481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:17.485912 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:17.486192 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:17.491463 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.491466 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.491457 11487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.491494 11481 server_base.cc:1034] running on GCE node
I20260923 22:13:17.491905 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:17.492106 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:17.493247 11481 hybrid_clock.cc:648] HybridClock initialized: now 1790201597493241 us; error 42 us; skew 500 ppm
I20260923 22:13:17.494463 11481 webserver.cc:460] Webserver started at http://127.1.132.130:42133/ using document root <none> and password file <none>
I20260923 22:13:17.494683 11481 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:17.494760 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:17.494889 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:17.495935 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "1fc78eb8cfa2408890bb795cac7bf62f"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "f4908f0432580ecb5549791c4eca3da3"
server_key_iv: "79ca542b0cd0a07569ea36f732cf59c8"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.496295 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "1fc78eb8cfa2408890bb795cac7bf62f"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "f4908f0432580ecb5549791c4eca3da3"
server_key_iv: "79ca542b0cd0a07569ea36f732cf59c8"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.497632 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:17.498431 11495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.498569 11481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:13:17.498646 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "1fc78eb8cfa2408890bb795cac7bf62f"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "f4908f0432580ecb5549791c4eca3da3"
server_key_iv: "79ca542b0cd0a07569ea36f732cf59c8"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.498726 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:17.502197 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:17.502467 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:17.502592 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:17.502832 11481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:17.503161 11481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:17.503196 11481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.503218 11481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:17.503230 11481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.508594 11481 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:38993
I20260923 22:13:17.508632 11608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:38993 every 8 connection(s)
I20260923 22:13:17.508982 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:13:17.511893 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.509685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:48199 (local address 127.1.132.190:35709)
0923 22:13:17.509733 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.509735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.509954 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:13:17.510463 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.510464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.510473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.510523 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.511644 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:17.511646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:17.511646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:17.511651 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:17.511669 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:17.511683 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:17.511733 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:17.511808 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:13:17.511823 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:13:17.512082 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.509674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35709 (local address 127.1.132.130:48199)
0923 22:13:17.509841 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:17.509875 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:17.510364 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:17.510591 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:17.510595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:17.510605 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:17.511497 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:17.511498 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:17.511527 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:17.511556 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:17.511736 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:17.511741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:17.511744 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:17.511813 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:13:17.511875 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260923 22:13:17.512563 11609 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35709
I20260923 22:13:17.512640 11609 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:17.512774 11609 heartbeater.cc:511] Master 127.1.132.190:35709 requested a full tablet report, sending...
I20260923 22:13:17.513089 11290 ts_manager.cc:194] Registered new tserver with Master: 1fc78eb8cfa2408890bb795cac7bf62f (127.1.132.130:38993)
I20260923 22:13:17.514993 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11481
I20260923 22:13:17.515076 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:17.515249 1554 external_mini_cluster.cc:1428] Setting key debaa52e187224e17f63533664e01789
I20260923 22:13:17.516022 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--builtin_ntp_servers=127.1.132.148:36689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:13:17.596742 11612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:17.596947 11612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:17.596969 11612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:17.598150 11612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:17.598191 11612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:17.598217 11612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:13:17.599572 11612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:17.599824 11612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:17.600126 11612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:17.605623 11618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.605618 11620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:17.605618 11617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:17.605973 11612 server_base.cc:1034] running on GCE node
I20260923 22:13:17.606235 11612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:17.606493 11612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:17.607666 11612 hybrid_clock.cc:648] HybridClock initialized: now 1790201597607649 us; error 42 us; skew 500 ppm
I20260923 22:13:17.608990 11612 webserver.cc:460] Webserver started at http://127.1.132.131:43793/ using document root <none> and password file <none>
I20260923 22:13:17.609251 11612 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:17.609304 11612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:17.609427 11612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:17.610530 11612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "9aeafb25ddd64862a00c4dae09732ac3"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "13abfab3f2b716f8eb2e50a2d3b760c9"
server_key_iv: "af59ed6fca520b5547a1a9f9d0c727f5"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.610862 11612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "9aeafb25ddd64862a00c4dae09732ac3"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "13abfab3f2b716f8eb2e50a2d3b760c9"
server_key_iv: "af59ed6fca520b5547a1a9f9d0c727f5"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.612219 11612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:13:17.613147 11626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.613358 11612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:17.613451 11612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9aeafb25ddd64862a00c4dae09732ac3"
format_stamp: "Formatted at 2026-09-23 22:13:17 on dist-test-slave-l4q5"
server_key: "13abfab3f2b716f8eb2e50a2d3b760c9"
server_key_iv: "af59ed6fca520b5547a1a9f9d0c727f5"
server_key_version: "encryptionkey@0"
I20260923 22:13:17.613516 11612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:17.617017 11612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:17.617287 11612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:17.617413 11612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:17.617601 11612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:17.617916 11612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:17.617949 11612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.617977 11612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:17.618003 11612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:17.623428 11612 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:36555
I20260923 22:13:17.623500 11739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:36555 every 8 connection(s)
I20260923 22:13:17.623818 11612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:17.626819 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.624618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:56403 (local address 127.1.132.190:35709)
0923 22:13:17.624671 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.624673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.624906 (+ 233us) server_negotiation.cc:407] Connection header received
0923 22:13:17.625405 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.625406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.625414 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.625471 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.626575 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:17.626576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:17.626576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:13:17.626581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:17.626597 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:13:17.626602 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:17.626652 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:17.626730 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:13:17.626746 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:13:17.626995 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:17.624596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35709 (local address 127.1.132.131:56403)
0923 22:13:17.624793 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:17.624826 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:17.625308 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:17.625542 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:17.625545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:17.625554 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:17.626428 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:13:17.626428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:17.626456 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:13:17.626484 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:17.626655 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:17.626659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:17.626663 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:17.626729 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:13:17.626791 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260923 22:13:17.627447 11740 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35709
I20260923 22:13:17.627523 11740 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:17.627655 11740 heartbeater.cc:511] Master 127.1.132.190:35709 requested a full tablet report, sending...
I20260923 22:13:17.628000 11290 ts_manager.cc:194] Registered new tserver with Master: 9aeafb25ddd64862a00c4dae09732ac3 (127.1.132.131:36555)
I20260923 22:13:17.630573 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11612
I20260923 22:13:17.630666 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:17.630815 1554 external_mini_cluster.cc:1428] Setting key 3981d099d89d3cd2c1047a88f99d4ae3
I20260923 22:13:17.631628 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:13:17.633352 11330 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:13:17.632899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.1.132.190:35709)
0923 22:13:17.632966 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:17.632968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:17.633086 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:13:17.633127 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:17.633128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:17.633134 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:17.633184 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:17.633289 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38960: BlockingRecv error: recv got EOF from 127.0.0.1:38960 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:13:17.633446 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11350
I20260923 22:13:17.637339 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11481
I20260923 22:13:17.642154 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11612
I20260923 22:13:17.646561 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11259
2026-09-23T22:13:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (535 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:13:17 dist-test-slave-l4q5 krb5kdc[11750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:13:17 dist-test-slave-l4q5 krb5kdc[11750](info): set up 2 sockets
Sep 23 22:13:17 dist-test-slave-l4q5 krb5kdc[11750](info): commencing operation
krb5kdc: starting...
W20260923 22:13:18.609038 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:13:18 dist-test-slave-l4q5 krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:13:18.730556 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36385
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:18.810186 11763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:18.810420 11763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:18.810446 11763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:18.811738 11763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:13:18.811800 11763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:18.811812 11763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:18.811825 11763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:13:18.811836 11763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:13:18.813503 11763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:18.814158 11763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:18.814397 11763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:18.819631 11769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:18.819650 11768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:18.819748 11771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:18.819671 11763 server_base.cc:1034] running on GCE node
I20260923 22:13:18.820093 11763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:18.820421 11763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:18.821609 11763 hybrid_clock.cc:648] HybridClock initialized: now 1790201598821590 us; error 31 us; skew 500 ppm
Sep 23 22:13:18 dist-test-slave-l4q5 krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:18.824447 11763 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:13:18.825178 11763 webserver.cc:460] Webserver started at http://127.1.132.190:41361/ using document root <none> and password file <none>
I20260923 22:13:18.825415 11763 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:18.825469 11763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:18.825601 11763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:18.826694 11763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "faffdd114d304f15b0106becd6b8755b"
format_stamp: "Formatted at 2026-09-23 22:13:18 on dist-test-slave-l4q5"
server_key: "6aa139ed75d2822447f1d5ef7c2c990e"
server_key_iv: "d9fdfc8fc5b3229acb23ec484dd95245"
server_key_version: "encryptionkey@0"
I20260923 22:13:18.827029 11763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "faffdd114d304f15b0106becd6b8755b"
format_stamp: "Formatted at 2026-09-23 22:13:18 on dist-test-slave-l4q5"
server_key: "6aa139ed75d2822447f1d5ef7c2c990e"
server_key_iv: "d9fdfc8fc5b3229acb23ec484dd95245"
server_key_version: "encryptionkey@0"
I20260923 22:13:18.828375 11763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:13:18.829303 11778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:18.829540 11763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:18.829639 11763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "faffdd114d304f15b0106becd6b8755b"
format_stamp: "Formatted at 2026-09-23 22:13:18 on dist-test-slave-l4q5"
server_key: "6aa139ed75d2822447f1d5ef7c2c990e"
server_key_iv: "d9fdfc8fc5b3229acb23ec484dd95245"
server_key_version: "encryptionkey@0"
I20260923 22:13:18.829717 11763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:18.852747 11763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:18.854847 11763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:18.855000 11763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:18.858942 11763 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36385
I20260923 22:13:18.859161 11830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36385 every 8 connection(s)
I20260923 22:13:18.859319 11763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:13:18.860375 11831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:18.862596 11831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: Bootstrap starting.
I20260923 22:13:18.863161 11831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:18.863380 11831 log.cc:824] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:18.864135 11831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: No bootstrap required, opened a new log
I20260923 22:13:18.865835 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11763
I20260923 22:13:18.865947 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:13:18.866123 1554 external_mini_cluster.cc:1428] Setting key 408b13c75ff8a80e6ddbffc55606b324
I20260923 22:13:18.866041 11831 raft_consensus.cc:348] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } }
I20260923 22:13:18.866237 11831 raft_consensus.cc:374] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:18.866274 11831 raft_consensus.cc:729] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faffdd114d304f15b0106becd6b8755b, State: Initialized, Role: FOLLOWER
I20260923 22:13:18.866432 11831 consensus_queue.cc:260] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } }
I20260923 22:13:18.866595 11831 raft_consensus.cc:388] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:13:18.866911 11831 raft_consensus.cc:482] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:13:18.866979 11831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:18.868353 11831 raft_consensus.cc:504] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } }
I20260923 22:13:18.868467 11831 leader_election.cc:302] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faffdd114d304f15b0106becd6b8755b; no voters:
I20260923 22:13:18.868651 11831 leader_election.cc:288] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:13:18.868703 11836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:18.868882 11836 raft_consensus.cc:686] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [term 1 LEADER]: Becoming Leader. State: Replica: faffdd114d304f15b0106becd6b8755b, State: Running, Role: LEADER
I20260923 22:13:18.868911 11831 sys_catalog.cc:564] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:13:18.869059 11836 consensus_queue.cc:237] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } }
I20260923 22:13:18.869407 11838 sys_catalog.cc:455] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faffdd114d304f15b0106becd6b8755b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } } }
I20260923 22:13:18.869529 11838 sys_catalog.cc:458] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:18.869416 11837 sys_catalog.cc:455] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [sys.catalog]: SysCatalogTable state changed. Reason: New leader faffdd114d304f15b0106becd6b8755b. Latest consensus state: current_term: 1 leader_uuid: "faffdd114d304f15b0106becd6b8755b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffdd114d304f15b0106becd6b8755b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36385 } } }
I20260923 22:13:18.870342 11837 sys_catalog.cc:458] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b [sys.catalog]: This master's current role is: LEADER
I20260923 22:13:18.870487 11845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:13:18.871282 11845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:13:18.872864 11845 catalog_manager.cc:1347] Generated new cluster ID: f77ed0f042b645bea5ba79b785e0c200
I20260923 22:13:18.872915 11845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:13:18 dist-test-slave-l4q5 krb5kdc[11750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:18.878579 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:18.866693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.1.132.190:36385)
0923 22:13:18.866888 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:18.866896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:18.866934 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:13:18.867072 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:18.867079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:18.867128 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:18.867323 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:18.868088 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:18.869384 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:18.870359 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:18.870973 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:18.871176 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:18.874068 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:18.874080 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:18.874080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:18.874095 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:18.876229 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:18.876677 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:18.876681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:18.876681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:18.876822 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:18.877029 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:18.877030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:18.877030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:18.877412 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:18.877572 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:18.878146 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 22:13:18.878307 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260923 22:13:18.890619 11845 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:13:18.891208 11845 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:13:18.894418 11845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: Generated new TSK 0
I20260923 22:13:18.894631 11845 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:13:18.928315 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:19.011420 11857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:19.011668 11857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:19.011703 11857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:19.013341 11857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:19.013418 11857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:19.013480 11857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:13:19.015841 11857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:19.016165 11857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:19.016430 11857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:19.021569 11865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.021591 11863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.021569 11862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:19.021768 11857 server_base.cc:1034] running on GCE node
I20260923 22:13:19.022159 11857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:19.022377 11857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:19.023527 11857 hybrid_clock.cc:648] HybridClock initialized: now 1790201599023514 us; error 28 us; skew 500 ppm
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:19.026064 11857 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:13:19.026503 11857 webserver.cc:460] Webserver started at http://127.1.132.129:40913/ using document root <none> and password file <none>
I20260923 22:13:19.026719 11857 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:19.026763 11857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:19.026883 11857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:19.027932 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "1fa7ec1c55834a12975ee4116847dde5"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "8c7cd401310a332d24f02df3d0b86dd6"
server_key_iv: "f4a783251ebd9d2ef51bc1980b4cd190"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.028282 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "1fa7ec1c55834a12975ee4116847dde5"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "8c7cd401310a332d24f02df3d0b86dd6"
server_key_iv: "f4a783251ebd9d2ef51bc1980b4cd190"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.029817 11857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:19.030637 11872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.030817 11857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:13:19.030911 11857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "1fa7ec1c55834a12975ee4116847dde5"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "8c7cd401310a332d24f02df3d0b86dd6"
server_key_iv: "f4a783251ebd9d2ef51bc1980b4cd190"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.030973 11857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:19.046211 11857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:19.048152 11857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:19.048354 11857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:19.048573 11857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:19.048887 11857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:19.048923 11857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.048954 11857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:19.048976 11857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.054351 11857 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:33361
I20260923 22:13:19.054399 11985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:33361 every 8 connection(s)
I20260923 22:13:19.054719 11857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:19.063759 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11857
I20260923 22:13:19.063853 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:13:19.064049 1554 external_mini_cluster.cc:1428] Setting key a656fe2b1b2019070eda07d9fa9247fc
I20260923 22:13:19.067044 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.055419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:33653 (local address 127.1.132.190:36385)
0923 22:13:19.055486 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.055489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.055665 (+ 176us) server_negotiation.cc:407] Connection header received
0923 22:13:19.056220 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.056222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.056239 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.056309 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:19.057298 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.057821 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.058848 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.059049 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.059192 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.063013 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:19.063015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:19.063015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:19.063024 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:19.064830 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.065809 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.065811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.065812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.065903 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.066276 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.066277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.066278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.066402 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:19.066468 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.066823 (+ 355us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.066943 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:13:19.067113 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.055390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36385 (local address 127.1.132.129:33653)
0923 22:13:19.055541 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.055578 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.056058 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.056373 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.056377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.056936 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:19.057133 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.057138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.057913 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.057914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.058750 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.058753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.059206 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.059210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.059332 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.060713 (+ 1381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:19.060723 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:19.062860 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:19.065009 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.065030 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.065034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.065689 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.065993 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.065994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.065995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.066178 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.066467 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:19.066474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:19.066683 (+ 209us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.066781 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.066931 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260923 22:13:19.067597 11986 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36385
I20260923 22:13:19.067699 11986 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:19.067891 11986 heartbeater.cc:511] Master 127.1.132.190:36385 requested a full tablet report, sending...
I20260923 22:13:19.068428 11795 ts_manager.cc:194] Registered new tserver with Master: 1fa7ec1c55834a12975ee4116847dde5 (127.1.132.129:33361)
I20260923 22:13:19.069135 11795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:33653
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:13:19.089339 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:19.170145 11991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:19.170367 11991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:19.170390 11991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:19.171634 11991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:19.171693 11991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:19.171722 11991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:13:19.173273 11991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:19.173507 11991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:19.173739 11991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:19.178795 11997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.178836 11999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.178879 11996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:19.178918 11991 server_base.cc:1034] running on GCE node
I20260923 22:13:19.179255 11991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:19.179514 11991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:19.180725 11991 hybrid_clock.cc:648] HybridClock initialized: now 1790201599180713 us; error 38 us; skew 500 ppm
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:19.183341 11991 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:13:19.183853 11991 webserver.cc:460] Webserver started at http://127.1.132.130:35011/ using document root <none> and password file <none>
I20260923 22:13:19.184072 11991 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:19.184115 11991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:19.184264 11991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:19.185319 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "b8f40aaf37644879bf117f11d4fa0f77"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "383b7a5e096ca57f1bee9a9305a46a67"
server_key_iv: "a922907b6d716b67934120586e4e7051"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.185636 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "b8f40aaf37644879bf117f11d4fa0f77"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "383b7a5e096ca57f1bee9a9305a46a67"
server_key_iv: "a922907b6d716b67934120586e4e7051"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.186909 11991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:13:19.187738 12006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.188066 11991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:13:19.188175 11991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "b8f40aaf37644879bf117f11d4fa0f77"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "383b7a5e096ca57f1bee9a9305a46a67"
server_key_iv: "a922907b6d716b67934120586e4e7051"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.188283 11991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:19.204495 11991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:19.207142 11991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:19.207327 11991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:19.207576 11991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:19.207954 11991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:19.208007 11991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.208048 11991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:19.208074 11991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.217670 11991 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:43637
I20260923 22:13:19.217722 12119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:43637 every 8 connection(s)
I20260923 22:13:19.218114 11991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:19.224406 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 11991
I20260923 22:13:19.224504 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:13:19.224689 1554 external_mini_cluster.cc:1428] Setting key 1211507423468f5531c4b0b92f8e404d
I20260923 22:13:19.227489 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.218950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:42319 (local address 127.1.132.190:36385)
0923 22:13:19.219002 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.219004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.219325 (+ 321us) server_negotiation.cc:407] Connection header received
0923 22:13:19.219966 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.219967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.219979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.220031 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:19.221000 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.221516 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.222481 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.222698 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.222754 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.224850 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:19.224851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:19.224852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:19.224860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:19.226192 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.226589 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.226590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.226591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.226654 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.226912 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.226913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.226913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.227019 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:19.227070 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.227357 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.227425 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:13:19.227731 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.218928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36385 (local address 127.1.132.130:42319)
0923 22:13:19.219173 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.219234 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.219860 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.220100 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.220104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.220714 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:19.220908 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.220913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.221600 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.221601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.222401 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.222403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.222770 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.222771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.222885 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.223344 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:19.223352 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:19.224767 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:19.226289 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.226295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.226296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.226527 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.226717 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.226718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.226719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.226853 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.227071 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:19.227075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:19.227233 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.227346 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.227505 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":138,"threads_started":1}
I20260923 22:13:19.228252 12120 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36385
I20260923 22:13:19.228350 12120 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:19.228518 12120 heartbeater.cc:511] Master 127.1.132.190:36385 requested a full tablet report, sending...
I20260923 22:13:19.228863 11795 ts_manager.cc:194] Registered new tserver with Master: b8f40aaf37644879bf117f11d4fa0f77 (127.1.132.130:43637)
I20260923 22:13:19.229223 11795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:42319
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:13:19.250269 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:13:19.330487 12125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:13:19.330745 12125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:13:19.330781 12125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:13:19.332257 12125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:13:19.332309 12125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:13:19.332337 12125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:13:19.333966 12125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:36385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:13:19.334211 12125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:13:19.334443 12125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:13:19.339743 12130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.339797 12131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:13:19.339743 12133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:13:19.339972 12125 server_base.cc:1034] running on GCE node
I20260923 22:13:19.340168 12125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:13:19.340474 12125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:13:19.341646 12125 hybrid_clock.cc:648] HybridClock initialized: now 1790201599341628 us; error 29 us; skew 500 ppm
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:13:19.344398 12125 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:13:19.344858 12125 webserver.cc:460] Webserver started at http://127.1.132.131:37031/ using document root <none> and password file <none>
I20260923 22:13:19.345060 12125 fs_manager.cc:359] Metadata directory not provided
I20260923 22:13:19.345104 12125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:13:19.345227 12125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:13:19.346346 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "fb59df70a3404f94877e3a77dc69f046"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "bc167a5e00cbd3d4cd81ca75f63ddaba"
server_key_iv: "e900f0556acf7a51abfc512afb151dd0"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.346678 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "fb59df70a3404f94877e3a77dc69f046"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "bc167a5e00cbd3d4cd81ca75f63ddaba"
server_key_iv: "e900f0556acf7a51abfc512afb151dd0"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.347910 12125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:13:19.348644 12140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.348850 12125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.348937 12125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "fb59df70a3404f94877e3a77dc69f046"
format_stamp: "Formatted at 2026-09-23 22:13:19 on dist-test-slave-l4q5"
server_key: "bc167a5e00cbd3d4cd81ca75f63ddaba"
server_key_iv: "e900f0556acf7a51abfc512afb151dd0"
server_key_version: "encryptionkey@0"
I20260923 22:13:19.348991 12125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:13:19.364573 12125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:13:19.366706 12125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:13:19.366823 12125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:13:19.367028 12125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:13:19.367290 12125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:13:19.367319 12125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.367341 12125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:13:19.367357 12125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:13:19.373072 12125 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:40805
I20260923 22:13:19.373131 12253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:40805 every 8 connection(s)
I20260923 22:13:19.373443 12125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:13:19.375160 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 12125
I20260923 22:13:19.375269 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:13:19.375443 1554 external_mini_cluster.cc:1428] Setting key 963c50742ae1f9fee7abe05fdc17f090
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[11750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:13:19.383953 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.374266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:44515 (local address 127.1.132.190:36385)
0923 22:13:19.374342 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.374344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.374467 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:13:19.375013 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.375015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.375029 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.375102 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:19.376240 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.376891 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.377976 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.378170 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.378249 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.380687 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:19.380689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:19.380689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:19.380699 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:19.382360 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.382919 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.382921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.382921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.382996 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.383301 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.383302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.383303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.383432 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:19.383494 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.383806 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.383882 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:13:19.384084 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.374188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:36385 (local address 127.1.132.131:44515)
0923 22:13:19.374350 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.374390 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.374880 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.375196 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.375201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.375872 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:13:19.376118 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.376125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.377001 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.377003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.377852 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.377855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.378286 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.378287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.378431 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.378967 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:13:19.378976 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:13:19.380538 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:13:19.382519 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.382529 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.382531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.382820 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.383087 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:13:19.383088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:13:19.383088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:13:19.383215 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:13:19.383496 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:13:19.383502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:13:19.383653 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.383746 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.383894 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260923 22:13:19.384594 12254 heartbeater.cc:347] Connected to a master server at 127.1.132.190:36385
I20260923 22:13:19.384773 12254 heartbeater.cc:464] Registering TS with master...
I20260923 22:13:19.384946 12254 heartbeater.cc:511] Master 127.1.132.190:36385 requested a full tablet report, sending...
I20260923 22:13:19.385423 11795 ts_manager.cc:194] Registered new tserver with Master: fb59df70a3404f94877e3a77dc69f046 (127.1.132.131:40805)
I20260923 22:13:19.385825 11795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:44515
I20260923 22:13:19.387882 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:13:19.396124 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.389432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.1.132.190:36385)
0923 22:13:19.389505 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.389506 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.389557 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:13:19.389588 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.389588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.389597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.389659 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:13:19.390256 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.390872 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.391670 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.391854 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.391910 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.393448 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:13:19.393450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:13:19.393450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:13:19.393460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:13:19.394941 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.395315 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.395316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.395317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.395380 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:13:19.395551 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:13:19.395551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:13:19.395552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:13:19.395711 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:13:19.395810 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.395935 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.396019 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:13:19.398087 11795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:13:19.399972 11795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:13:19.406605 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33361 (local address 127.0.0.1:46300)
0923 22:13:19.403082 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.403091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.403151 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.403558 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.403559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.403561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:19.403599 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.403600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404820 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.404821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.405659 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.405661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406324 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.406398 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406410 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.406468 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.406522 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":63,"threads_started":1}
I20260923 22:13:19.406627 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.1.132.129:33361)
0923 22:13:19.402985 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.402989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.403424 (+ 435us) server_negotiation.cc:407] Connection header received
0923 22:13:19.403468 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.403470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.403484 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.403560 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:19.403694 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404735 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.405751 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406242 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406310 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406371 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.406488 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.406528 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260923 22:13:19.406939 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.1.132.130:43637)
0923 22:13:19.402838 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.402843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.402949 (+ 106us) server_negotiation.cc:407] Connection header received
0923 22:13:19.403062 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.403065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.403080 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.403151 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:19.403357 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404234 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.405798 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406529 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406595 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406650 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.406825 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.406868 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260923 22:13:19.406839 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40805 (local address 127.0.0.1:41956)
0923 22:13:19.402828 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.402879 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.402967 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.403244 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.403245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.403247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:19.403305 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.403307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404628 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.404629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.405695 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.405697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406625 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.406690 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406694 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.406723 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.406765 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":56,"threads_started":1}
I20260923 22:13:19.407137 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.1.132.131:40805)
0923 22:13:19.403015 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.403019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.403029 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:13:19.403067 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.403069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.403084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.403168 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:19.403374 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404525 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.405797 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406528 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406923 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406987 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.407014 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.407060 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":108,"threads_started":1}
I20260923 22:13:19.406919 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.402497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:43637 (local address 127.0.0.1:51784)
0923 22:13:19.402884 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.402893 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.402977 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.403163 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.403166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.403168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:19.403276 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.403279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.404325 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.404325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.405695 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.405697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.406677 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.406678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.406741 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.406744 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.406790 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.406824 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260923 22:13:19.407910 12188 tablet_service.cc:1467] Processing CreateTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 (DEFAULT_TABLE table=test-table [id=596a6ea30c614577a6cfc14338c9d200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:19.407889 12054 tablet_service.cc:1467] Processing CreateTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 (DEFAULT_TABLE table=test-table [id=596a6ea30c614577a6cfc14338c9d200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:19.407887 11920 tablet_service.cc:1467] Processing CreateTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 (DEFAULT_TABLE table=test-table [id=596a6ea30c614577a6cfc14338c9d200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:13:19.408269 12188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5d02491c8f46d0932f07dbac47d6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:19.408273 11920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5d02491c8f46d0932f07dbac47d6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:19.408274 12054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5d02491c8f46d0932f07dbac47d6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:13:19.410846 12271 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Bootstrap starting.
I20260923 22:13:19.411466 12273 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Bootstrap starting.
I20260923 22:13:19.411595 12271 tablet_bootstrap.cc:654] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:19.412143 12273 tablet_bootstrap.cc:654] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:19.412159 12271 log.cc:824] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:19.412400 12272 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: Bootstrap starting.
I20260923 22:13:19.412829 12273 log.cc:824] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:19.413141 12271 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: No bootstrap required, opened a new log
I20260923 22:13:19.413159 12272 tablet_bootstrap.cc:654] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: Neither blocks nor log segments found. Creating new log.
I20260923 22:13:19.413208 12271 ts_tablet_manager.cc:1397] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:13:19.413614 12273 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: No bootstrap required, opened a new log
I20260923 22:13:19.413630 12272 log.cc:824] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:13:19.413751 12273 ts_tablet_manager.cc:1397] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:13:19.414294 12272 tablet_bootstrap.cc:492] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: No bootstrap required, opened a new log
I20260923 22:13:19.414355 12272 ts_tablet_manager.cc:1397] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:13:19.415182 12271 raft_consensus.cc:348] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.415369 12271 raft_consensus.cc:374] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:19.415406 12271 raft_consensus.cc:729] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb59df70a3404f94877e3a77dc69f046, State: Initialized, Role: FOLLOWER
I20260923 22:13:19.415530 12271 consensus_queue.cc:260] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.415711 12273 raft_consensus.cc:348] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.415834 12254 heartbeater.cc:503] Master 127.1.132.190:36385 was elected leader, sending a full tablet report...
I20260923 22:13:19.415863 12273 raft_consensus.cc:374] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:19.415899 12273 raft_consensus.cc:729] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f40aaf37644879bf117f11d4fa0f77, State: Initialized, Role: FOLLOWER
I20260923 22:13:19.416004 12273 consensus_queue.cc:260] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.415927 12272 raft_consensus.cc:348] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.416096 12271 ts_tablet_manager.cc:1428] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:13:19.416078 12272 raft_consensus.cc:374] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:13:19.416225 12272 raft_consensus.cc:729] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa7ec1c55834a12975ee4116847dde5, State: Initialized, Role: FOLLOWER
I20260923 22:13:19.416288 12273 ts_tablet_manager.cc:1428] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:13:19.416350 12272 consensus_queue.cc:260] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.416405 12120 heartbeater.cc:503] Master 127.1.132.190:36385 was elected leader, sending a full tablet report...
I20260923 22:13:19.416712 11986 heartbeater.cc:503] Master 127.1.132.190:36385 was elected leader, sending a full tablet report...
I20260923 22:13:19.416774 12272 ts_tablet_manager.cc:1428] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:13:19.467227 12279 raft_consensus.cc:482] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:13:19.467353 12279 raft_consensus.cc:504] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.467741 12279 leader_election.cc:288] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb59df70a3404f94877e3a77dc69f046 (127.1.132.131:40805), b8f40aaf37644879bf117f11d4fa0f77 (127.1.132.130:43637)
W20260923 22:13:19.468711 12121 tablet.cc:2369] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:13:19.470770 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.467998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58435 (local address 127.1.132.131:40805)
0923 22:13:19.468038 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.468040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.468056 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:13:19.468174 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.468175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.468213 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.468281 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:19.468439 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.468979 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470017 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.470448 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470516 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.470526 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.470661 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.470696 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:13:19.470773 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.467927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:40805 (local address 127.1.132.129:58435)
0923 22:13:19.467994 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.468006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.468073 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.468290 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.468291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.468293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:19.468372 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.468373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.469052 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.469054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.469911 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.469914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.470520 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.470599 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.470602 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.470655 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.470698 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":30}
I20260923 22:13:19.470844 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.468018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:41311 (local address 127.1.132.130:43637)
0923 22:13:19.468064 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.468066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.468230 (+ 164us) server_negotiation.cc:407] Connection header received
0923 22:13:19.468274 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.468275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.468283 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.468377 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:13:19.468525 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.469135 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470089 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.470542 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470602 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.470611 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.470747 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.470779 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:13:19.470880 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.467927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:43637 (local address 127.1.132.129:41311)
0923 22:13:19.468127 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.468135 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:13:19.468230 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:13:19.468400 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:13:19.468402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:13:19.468404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:13:19.468464 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.468466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.469212 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.469213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.470016 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:13:19.470019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.470610 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.470611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:13:19.470683 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.470688 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:13:19.470733 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:13:19.470773 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":21,"thread_start_us":110,"threads_started":1}
I20260923 22:13:19.471163 12208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5d02491c8f46d0932f07dbac47d6e0" candidate_uuid: "1fa7ec1c55834a12975ee4116847dde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb59df70a3404f94877e3a77dc69f046" is_pre_election: true
I20260923 22:13:19.471154 12072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5d02491c8f46d0932f07dbac47d6e0" candidate_uuid: "1fa7ec1c55834a12975ee4116847dde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f40aaf37644879bf117f11d4fa0f77" is_pre_election: true
I20260923 22:13:19.471302 12072 raft_consensus.cc:2454] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa7ec1c55834a12975ee4116847dde5 in term 0.
I20260923 22:13:19.471302 12208 raft_consensus.cc:2454] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa7ec1c55834a12975ee4116847dde5 in term 0.
I20260923 22:13:19.471509 11875 leader_election.cc:302] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa7ec1c55834a12975ee4116847dde5, b8f40aaf37644879bf117f11d4fa0f77; no voters:
I20260923 22:13:19.471652 12279 raft_consensus.cc:2781] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:13:19.471711 12279 raft_consensus.cc:482] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:13:19.471733 12279 raft_consensus.cc:3037] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:19.472468 12279 raft_consensus.cc:504] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.472600 12279 leader_election.cc:288] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [CANDIDATE]: Term 1 election: Requested vote from peers fb59df70a3404f94877e3a77dc69f046 (127.1.132.131:40805), b8f40aaf37644879bf117f11d4fa0f77 (127.1.132.130:43637)
I20260923 22:13:19.472775 12208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5d02491c8f46d0932f07dbac47d6e0" candidate_uuid: "1fa7ec1c55834a12975ee4116847dde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb59df70a3404f94877e3a77dc69f046"
I20260923 22:13:19.472839 12072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5d02491c8f46d0932f07dbac47d6e0" candidate_uuid: "1fa7ec1c55834a12975ee4116847dde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f40aaf37644879bf117f11d4fa0f77"
I20260923 22:13:19.472852 12208 raft_consensus.cc:3037] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:19.472914 12072 raft_consensus.cc:3037] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:13:19.473541 12208 raft_consensus.cc:2454] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa7ec1c55834a12975ee4116847dde5 in term 1.
I20260923 22:13:19.473587 12072 raft_consensus.cc:2454] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa7ec1c55834a12975ee4116847dde5 in term 1.
I20260923 22:13:19.473721 11875 leader_election.cc:302] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa7ec1c55834a12975ee4116847dde5, b8f40aaf37644879bf117f11d4fa0f77; no voters:
I20260923 22:13:19.473816 12279 raft_consensus.cc:2781] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:13:19.473950 12279 raft_consensus.cc:686] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [term 1 LEADER]: Becoming Leader. State: Replica: 1fa7ec1c55834a12975ee4116847dde5, State: Running, Role: LEADER
I20260923 22:13:19.474036 12279 consensus_queue.cc:237] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } }
I20260923 22:13:19.474850 11794 catalog_manager.cc:5495] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fa7ec1c55834a12975ee4116847dde5 (127.1.132.129). New cstate: current_term: 1 leader_uuid: "1fa7ec1c55834a12975ee4116847dde5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa7ec1c55834a12975ee4116847dde5" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 33361 } health_report { overall_health: HEALTHY } } }
I20260923 22:13:19.480159 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.477838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.1.132.129:33361)
0923 22:13:19.477888 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:13:19.477890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:13:19.477897 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:13:19.477928 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:13:19.477930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:13:19.477962 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:13:19.478020 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:13:19.478152 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.478639 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.479399 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:13:19.479619 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:13:19.479672 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:13:19.479818 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:13:19.479933 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:13:19.480003 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:13:19.480039 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 22:13:19.480061 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:13:19.482036 12208 raft_consensus.cc:1261] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 1 FOLLOWER]: Refusing update from remote peer 1fa7ec1c55834a12975ee4116847dde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:19.482036 12072 raft_consensus.cc:1261] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 1 FOLLOWER]: Refusing update from remote peer 1fa7ec1c55834a12975ee4116847dde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:13:19.482381 12281 consensus_queue.cc:1035] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f40aaf37644879bf117f11d4fa0f77" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 43637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:19.482514 12279 consensus_queue.cc:1035] T ce5d02491c8f46d0932f07dbac47d6e0 P 1fa7ec1c55834a12975ee4116847dde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb59df70a3404f94877e3a77dc69f046" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 40805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:13:19.485134 12285 mvcc.cc:205] Tried to move back new op lower bound from 7332665751475417088 to 7332665751446294528. Current Snapshot: MvccSnapshot[applied={T|T < 7332665751475417088}]
I20260923 22:13:19.485755 12286 mvcc.cc:205] Tried to move back new op lower bound from 7332665751475417088 to 7332665751446294528. Current Snapshot: MvccSnapshot[applied={T|T < 7332665751475417088}]
I20260923 22:13:19.487263 11794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:19.487389 11794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:13:19.488299 11794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faffdd114d304f15b0106becd6b8755b: Sending DeleteTablet for 3 replicas of tablet ce5d02491c8f46d0932f07dbac47d6e0
I20260923 22:13:19.488667 12054 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:19 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:51784
I20260923 22:13:19.488672 11920 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:19 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:46300
I20260923 22:13:19.488699 12188 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5d02491c8f46d0932f07dbac47d6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:13:19 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:41956
I20260923 22:13:19.489037 12296 tablet_replica.cc:330] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: stopping tablet replica
I20260923 22:13:19.489156 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11857
I20260923 22:13:19.489197 12296 raft_consensus.cc:2229] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:13:19.489318 12296 raft_consensus.cc:2258] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:19.489744 12295 tablet_replica.cc:330] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: stopping tablet replica
I20260923 22:13:19.489856 12295 raft_consensus.cc:2229] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:13:19.489997 12295 raft_consensus.cc:2258] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:13:19.490128 12296 ts_tablet_manager.cc:1905] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:13:19.490438 12295 ts_tablet_manager.cc:1905] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:13:19.492115 12295 ts_tablet_manager.cc:1918] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:13:19.492218 12295 log.cc:1196] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/ce5d02491c8f46d0932f07dbac47d6e0
I20260923 22:13:19.492220 12296 ts_tablet_manager.cc:1918] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:13:19.492300 12296 log.cc:1196] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/ce5d02491c8f46d0932f07dbac47d6e0
I20260923 22:13:19.492475 12295 ts_tablet_manager.cc:1939] T ce5d02491c8f46d0932f07dbac47d6e0 P b8f40aaf37644879bf117f11d4fa0f77: Deleting consensus metadata
I20260923 22:13:19.492583 12296 ts_tablet_manager.cc:1939] T ce5d02491c8f46d0932f07dbac47d6e0 P fb59df70a3404f94877e3a77dc69f046: Deleting consensus metadata
I20260923 22:13:19.492931 11781 catalog_manager.cc:4841] TS b8f40aaf37644879bf117f11d4fa0f77 (127.1.132.130:43637): tablet ce5d02491c8f46d0932f07dbac47d6e0 (table test-table [id=596a6ea30c614577a6cfc14338c9d200]) successfully deleted
I20260923 22:13:19.492980 11782 catalog_manager.cc:4841] TS fb59df70a3404f94877e3a77dc69f046 (127.1.132.131:40805): tablet ce5d02491c8f46d0932f07dbac47d6e0 (table test-table [id=596a6ea30c614577a6cfc14338c9d200]) successfully deleted
W20260923 22:13:19.495810 11781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:33361 (error 108)
I20260923 22:13:19.495944 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11991
I20260923 22:13:19.496903 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:13:19.496762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:33361 (local address 127.0.0.1:46316)
0923 22:13:19.496826 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:13:19.496867 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:33361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260923 22:13:19.497027 11781 catalog_manager.cc:4584] TS 1fa7ec1c55834a12975ee4116847dde5 (127.1.132.129:33361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce5d02491c8f46d0932f07dbac47d6e0: Network error: Client connection negotiation failed: client connection to 127.1.132.129:33361: connect: Connection refused (error 111)
I20260923 22:13:19.500326 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 12125
I20260923 22:13:19.504905 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 11763
2026-09-23T22:13:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1859 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[12300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[12300](info): set up 2 sockets
Sep 23 22:13:19 dist-test-slave-l4q5 krb5kdc[12300](info): commencing operation
krb5kdc: starting...
W20260923 22:13:20.472883 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:13:20 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:13:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangeradmin_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangeradmin_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangerlookup_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangerlookup_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
I20260923 22:13:20.643710 1554 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1S3mcj/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/postgres -l logfile start
2026-09-23 22:13:23.540 UTC [12330] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:13:23.540 UTC [12330] LOG: listening on IPv4 address "127.1.132.148", port 46169
2026-09-23 22:13:23.545 UTC [12330] LOG: listening on Unix socket "/tmp/.s.PGSQL.46169"
2026-09-23 22:13:23.567 UTC [12333] LOG: database system was shut down at 2026-09-23 22:13:21 UTC
2026-09-23 22:13:23.571 UTC [12330] LOG: database system is ready to accept connections
I20260923 22:13:24.440797 1554 mini_postgres.cc:96] Postgres bound to 46169
2026-09-23 22:13:24.446 UTC [12341] FATAL: database "slave" does not exist
127.1.132.148:46169 - accepting connections
I20260923 22:13:24.447052 1554 mini_ranger.cc:161] Starting Ranger...
I20260923 22:13:24.463075 1554 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:13:25.036898 1554 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:13:25.036968 1554 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin
2026-09-23 22:13:25,322 [I] DB FLAVOR :POSTGRES
2026-09-23 22:13:25,323 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:13:25,323 [I] Checking connection..
2026-09-23 22:13:25,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:13:25,569 [I] Checking connection passed.
2026-09-23 22:13:25,569 [I] --------- Verifying version history table ---------
2026-09-23 22:13:25,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:25,843 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:13:25,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:26,082 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:13:26,082 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:13:26,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:13:26.344 UTC [12405] WARNING: there is no transaction in progress
2026-09-23 22:13:26,350 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:13:26,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:13:26,618 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:13:26,618 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:13:26,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:13:26,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:13:27,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l4q5.c.gcp-upstream.internal','N') ;"
2026-09-23 22:13:27,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:13:27,626 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:13:27,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:13:27,897 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:13:27,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:13:28,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:13:28,395 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:13:28,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:13:29.440 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.457 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.476 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.494 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.511 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.529 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.590 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.598 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.614 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.625 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.636 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.653 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.665 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.672 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.681 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:29.688 UTC [12513] WARNING: there is no transaction in progress
2026-09-23 22:13:30,487 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:13:30,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:13:30,747 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:13:30,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:13:30,997 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:13:30,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:13:31,258 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:13:31,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:13:31,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:13:31,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l4q5.c.gcp-upstream.internal';"
2026-09-23 22:13:32,016 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:13:32,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:13:32,258 [I] DB_PATCHES have already been applied
I20260923 22:13:32.265008 1554 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1S3mcj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:13:32.265170 1554 mini_ranger.cc:193] Using host: 127.1.132.148
I20260923 22:13:32.268060 1554 mini_ranger.cc:227] Ranger admin URL: http://127.1.132.148:40761
Sep 23, 2026 10:13:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:13:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:13:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:13:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:13:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:13:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:13:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:13:34.301793 1554 mini_ranger.cc:160] Time spent starting Ranger: real 9.855s user 0.003s sys 0.007s
Sep 23, 2026 10:13:53 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:13:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:13:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:13:53 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:13:54 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:13:55.122114 1554 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangerkms_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/rangerkms_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:13:55 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:13:55.238056 1554 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:13:55.247573 1554 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:13:56.349188 1554 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:13:56.349261 1554 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms
I20260923 22:13:56.349280 1554 mini_ranger_kms.cc:227] Using postgres at 127.1.132.148:46169
2026-09-23 22:13:56,517 [I] DB FLAVOR :POSTGRES
2026-09-23 22:13:56,518 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:13:56,518 [I] Checking connection
2026-09-23 22:13:56,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:13:56,785 [I] connection success
2026-09-23 22:13:56,785 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:13:56,785 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:13:56,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:13:57,020 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:13:57,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:13:57,020 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:13:57,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:46169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:13:57,304 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:13:57.813895 1554 mini_ranger_kms.cc:310] Created kms service
I20260923 22:13:57.998347 1554 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:13:58.045431 1554 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:13:58 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:13:58.452123 1554 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:13:58.452268 1554 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms:/tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:13:58.452291 1554 mini_ranger_kms.cc:240] Using host: 127.1.132.148
I20260923 22:13:58.455317 1554 mini_ranger_kms.cc:276] Ranger KMS PID: 12694
I20260923 22:13:58.455451 1554 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.132.148:36695
Sep 23, 2026 10:13:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:13:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:13:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:13:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:14:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:14:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:14:00.518678 1554 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.281s user 0.010s sys 0.142s
I20260923 22:14:00.518774 1554 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:14:00.518854 1554 mini_ranger_kms.cc:401] 127.1.132.148:36695/kms/v1/keys
Sep 23 22:14:00 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:14:01 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
Sep 23 22:14:01 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
Sep 23 22:14:04 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:14:04.176523 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:43623
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:43623
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:36695/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:04.267349 12745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:04.267576 12745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:04.267599 12745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:04.268880 12745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:04.268931 12745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:04.268945 12745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:04.268956 12745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:04.268968 12745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:04.270581 12745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.132.148:36695/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:43623
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:04.270805 12745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:04.271041 12745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:04.275920 12750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:04.275920 12751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:04.276229 12745 server_base.cc:1034] running on GCE node
W20260923 22:14:04.276381 12753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:04.276652 12745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:04.276993 12745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:04.278173 12745 hybrid_clock.cc:648] HybridClock initialized: now 1790201644278156 us; error 34 us; skew 500 ppm
Sep 23 22:14:04 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:04.281342 12745 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:14:04.281865 12745 webserver.cc:460] Webserver started at http://127.1.132.190:33973/ using document root <none> and password file <none>
I20260923 22:14:04.282081 12745 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:04.282121 12745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:04.282250 12745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:04 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:04.335234 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "86e4795f78944574baaf086fb1646e4e"
format_stamp: "Formatted at 2026-09-23 22:14:04 on dist-test-slave-l4q5"
server_key: "171ff23a116557b3c0f28e8e4722b287"
server_key_iv: "8b81cb6abe1a62192824461dda3bc654"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:04.335628 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "86e4795f78944574baaf086fb1646e4e"
format_stamp: "Formatted at 2026-09-23 22:14:04 on dist-test-slave-l4q5"
server_key: "171ff23a116557b3c0f28e8e4722b287"
server_key_iv: "8b81cb6abe1a62192824461dda3bc654"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:04.337112 12745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:14:04.352869 12760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:04.353135 12745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:04.353279 12745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "86e4795f78944574baaf086fb1646e4e"
format_stamp: "Formatted at 2026-09-23 22:14:04 on dist-test-slave-l4q5"
server_key: "171ff23a116557b3c0f28e8e4722b287"
server_key_iv: "8b81cb6abe1a62192824461dda3bc654"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:04.353418 12745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:04.381408 12745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:04.383306 12745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:04.383460 12745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:04.388348 12745 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:43623
I20260923 22:14:04.388365 12812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:43623 every 8 connection(s)
I20260923 22:14:04.388836 12745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:04.389611 12813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:04.392685 12813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: Bootstrap starting.
I20260923 22:14:04.392889 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 12745
I20260923 22:14:04.392983 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:14:04.393160 1554 external_mini_cluster.cc:1428] Setting key 3d35d8103b4f7d99ead8a4a46d0898ad
I20260923 22:14:04.393342 12813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:04.394224 12813 log.cc:824] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:04.395022 12813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: No bootstrap required, opened a new log
I20260923 22:14:04.397312 12813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } }
I20260923 22:14:04.397466 12813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:04.397490 12813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e4795f78944574baaf086fb1646e4e, State: Initialized, Role: FOLLOWER
I20260923 22:14:04.397599 12813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } }
I20260923 22:14:04.397671 12813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:04.397701 12813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:04.397742 12813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:04.398357 12813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } }
I20260923 22:14:04.398465 12813 leader_election.cc:302] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e4795f78944574baaf086fb1646e4e; no voters:
I20260923 22:14:04.398626 12813 leader_election.cc:288] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:04.398679 12818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:04.398898 12818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [term 1 LEADER]: Becoming Leader. State: Replica: 86e4795f78944574baaf086fb1646e4e, State: Running, Role: LEADER
I20260923 22:14:04.398902 12813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:04.399070 12818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } }
I20260923 22:14:04.399447 12818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e4795f78944574baaf086fb1646e4e. Latest consensus state: current_term: 1 leader_uuid: "86e4795f78944574baaf086fb1646e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } } }
I20260923 22:14:04.399526 12818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:04.400370 12825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:04.400898 12825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:04.400935 12819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e4795f78944574baaf086fb1646e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e4795f78944574baaf086fb1646e4e" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 43623 } } }
I20260923 22:14:04.401095 12819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:04.401203 12813 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 23 22:14:04 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:04.405675 12825 catalog_manager.cc:1347] Generated new cluster ID: 42d7706b27c14a73b12eab7ccb1fd97d
I20260923 22:14:04.405748 12825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:04.409312 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:04.393669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.1.132.190:43623)
0923 22:14:04.394282 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:04.394291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:04.394323 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:14:04.394818 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:04.394827 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:04.394900 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:04.395050 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:04.395677 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:04.397111 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:04.398142 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:04.398768 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:04.398921 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:04.403340 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:04.403423 (+ 83us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:04.403423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:04.403512 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:04.406549 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:04.407114 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:04.407118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:04.407119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:04.407305 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:04.407714 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:04.407715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:04.407716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:04.408051 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:04.408670 (+ 619us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:04.408750 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:14:04.408903 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":508,"threads_started":1}
I20260923 22:14:04.416553 12825 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:04.417289 12825 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:04.426563 12825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: Generated new TSK 0
I20260923 22:14:04.426781 12825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:14:05.176523 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:36695/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:05.299930 12858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:05.300271 12858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:05.300354 12858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:05.302268 12858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:05.302342 12858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:05.302386 12858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:14:05.304209 12858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.132.148:36695/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:05.304478 12858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:05.304713 12858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:05.310247 12864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.310525 12863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.310247 12866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:05.311530 12858 server_base.cc:1034] running on GCE node
I20260923 22:14:05.311770 12858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:05.312098 12858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:05.313270 12858 hybrid_clock.cc:648] HybridClock initialized: now 1790201645313248 us; error 33 us; skew 500 ppm
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:05.316566 12858 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:14:05.317137 12858 webserver.cc:460] Webserver started at http://127.1.132.129:43157/ using document root <none> and password file <none>
I20260923 22:14:05.317377 12858 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:05.317425 12858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:05.317550 12858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:05.331189 12858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "481db82ea94f4e3c81fb3807b82c68cc"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "53bec6e4c4fad13738fd006a5552b7bd"
server_key_iv: "fabc6e42b78dee207ba9209ac743bebb"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.331615 12858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "481db82ea94f4e3c81fb3807b82c68cc"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "53bec6e4c4fad13738fd006a5552b7bd"
server_key_iv: "fabc6e42b78dee207ba9209ac743bebb"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.337675 12858 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260923 22:14:05.353528 12875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.353765 12858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:05.353854 12858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "481db82ea94f4e3c81fb3807b82c68cc"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "53bec6e4c4fad13738fd006a5552b7bd"
server_key_iv: "fabc6e42b78dee207ba9209ac743bebb"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.353960 12858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:05.383751 12858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:05.386308 12858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:05.386543 12858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:05.386819 12858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:05.387250 12858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:05.387343 12858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.387458 12858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:05.387528 12858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.394073 12858 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:37959
I20260923 22:14:05.394582 12858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:05.395742 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 12858
I20260923 22:14:05.395856 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:14:05.396039 1554 external_mini_cluster.cc:1428] Setting key 7994ecceeed0fb1d12d72a407f789d97
I20260923 22:14:05.398586 12988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:37959 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:05.422602 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.399275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43623 (local address 127.1.132.129:34977)
0923 22:14:05.404956 (+ 5681us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.405030 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.405708 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.412913 (+ 7205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.412920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.413216 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:05.413527 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.413534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.414665 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.414667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.416023 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.416027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.416588 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.416589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.416702 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.417373 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:05.417385 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:05.419026 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:05.420992 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.421002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.421005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.421310 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.421514 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.421515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.421515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.421642 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.421901 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:05.421907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:05.422071 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.422169 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.422341 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5620,"thread_start_us":99,"threads_started":1}
I20260923 22:14:05.423367 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.399640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:34977 (local address 127.1.132.190:43623)
0923 22:14:05.412300 (+ 12660us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.412303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.412320 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:14:05.412576 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.412578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.412601 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.412741 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:05.413669 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.414535 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.416132 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.416459 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.416500 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.419150 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:05.419152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:05.419152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:05.419160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:05.420898 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.421389 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.421391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.421391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.421454 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.421695 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.421696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.421696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.421825 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:05.421871 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.423161 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.423254 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12617,"thread_start_us":94,"threads_started":1}
I20260923 22:14:05.423839 12989 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43623
I20260923 22:14:05.423935 12989 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:05.424131 12989 heartbeater.cc:511] Master 127.1.132.190:43623 requested a full tablet report, sending...
I20260923 22:14:05.424727 12777 ts_manager.cc:194] Registered new tserver with Master: 481db82ea94f4e3c81fb3807b82c68cc (127.1.132.129:37959)
I20260923 22:14:05.425601 12777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:34977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:14:05.435247 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:36695/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:14:05.531870 12997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:05.532080 12997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:05.532114 12997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:05.533711 12997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:05.533818 12997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:05.533879 12997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:14:05.535974 12997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.132.148:36695/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:05.536350 12997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:05.536648 12997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:05.542176 13005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.542486 13003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.543298 13002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:05.543612 12997 server_base.cc:1034] running on GCE node
I20260923 22:14:05.543831 12997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:05.544062 12997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:05.545235 12997 hybrid_clock.cc:648] HybridClock initialized: now 1790201645545221 us; error 25 us; skew 500 ppm
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:05.547761 12997 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:14:05.548265 12997 webserver.cc:460] Webserver started at http://127.1.132.130:34333/ using document root <none> and password file <none>
I20260923 22:14:05.548485 12997 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:05.548533 12997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:05.548662 12997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:05.569234 12997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "bea1792640fb406e87d0242a66668927"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "ae79c81905c43b4997e0757ec21f838d"
server_key_iv: "8368bdaf0d4658d8353857114f9f25f7"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.569703 12997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "bea1792640fb406e87d0242a66668927"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "ae79c81905c43b4997e0757ec21f838d"
server_key_iv: "8368bdaf0d4658d8353857114f9f25f7"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.573583 12997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 22:14:05.588599 13013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.588766 12997 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260923 22:14:05.588855 12997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "bea1792640fb406e87d0242a66668927"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "ae79c81905c43b4997e0757ec21f838d"
server_key_iv: "8368bdaf0d4658d8353857114f9f25f7"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.588959 12997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:05.601944 12997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:05.603760 12997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:05.603902 12997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:05.604117 12997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:05.604471 12997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:05.604508 12997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.604543 12997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:05.604563 12997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.611143 12997 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:38199
I20260923 22:14:05.611500 12997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:14:05.611459 13126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:38199 every 8 connection(s)
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:05.618508 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 12997
I20260923 22:14:05.618618 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:14:05.618818 1554 external_mini_cluster.cc:1428] Setting key 8453e2332fee1163bdca5f54e835a9a7
I20260923 22:14:05.621999 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.612293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:60817 (local address 127.1.132.190:43623)
0923 22:14:05.612342 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.612344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.612718 (+ 374us) server_negotiation.cc:407] Connection header received
0923 22:14:05.613432 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.613434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.613451 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.613523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:05.614154 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.614924 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.615988 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.616330 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.616403 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.618812 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:05.618814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:05.618814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:05.618823 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:05.620438 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.620957 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.620959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.620959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.621022 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.621305 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.621307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.621307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.621451 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:05.621518 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.621864 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.621933 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:14:05.622234 13129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.612281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43623 (local address 127.1.132.130:60817)
0923 22:14:05.612583 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.612626 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.613311 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.613594 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.613598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.613840 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:05.614053 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.614058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.615026 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.615027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.615895 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.615897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.616418 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.616419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.616528 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.617427 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:05.617435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:05.618705 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:05.620519 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.620526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.620529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.620851 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.621095 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.621095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.621096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.621233 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.621531 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:05.621537 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:05.621707 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.621839 (+ 132us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.622000 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":145,"threads_started":1}
I20260923 22:14:05.622690 13127 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43623
I20260923 22:14:05.622794 13127 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:05.622969 13127 heartbeater.cc:511] Master 127.1.132.190:43623 requested a full tablet report, sending...
I20260923 22:14:05.623314 12777 ts_manager.cc:194] Registered new tserver with Master: bea1792640fb406e87d0242a66668927 (127.1.132.130:38199)
I20260923 22:14:05.623709 12777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:60817
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:14:05.649699 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:36695/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:05.746948 13132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:05.747282 13132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:05.747323 13132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:05.748983 13132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:05.749048 13132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:05.749089 13132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:14:05.751444 13132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.132.148:36695/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:43623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:05.751768 13132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:05.752056 13132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:05.758745 13137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.759197 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:05.760360 13140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:05.760828 13132 server_base.cc:1034] running on GCE node
I20260923 22:14:05.761255 13132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:05.761596 13132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:05.763549 13132 hybrid_clock.cc:648] HybridClock initialized: now 1790201645763518 us; error 45 us; skew 500 ppm
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:05.767143 13132 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:14:05.767757 13132 webserver.cc:460] Webserver started at http://127.1.132.131:38449/ using document root <none> and password file <none>
I20260923 22:14:05.768020 13132 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:05.768088 13132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:05.768252 13132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:05.784592 13132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "eccf426e0dab45a0842eacd7f542caa5"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "907b630186eb7311f10d82a7d5bc2f3d"
server_key_iv: "0c53c556a7ce680c818b481dba62d4e8"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.785112 13132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "eccf426e0dab45a0842eacd7f542caa5"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "907b630186eb7311f10d82a7d5bc2f3d"
server_key_iv: "0c53c556a7ce680c818b481dba62d4e8"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.786957 13132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 22:14:05.802101 13147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.802306 13132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 22:14:05.802407 13132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "eccf426e0dab45a0842eacd7f542caa5"
format_stamp: "Formatted at 2026-09-23 22:14:05 on dist-test-slave-l4q5"
server_key: "907b630186eb7311f10d82a7d5bc2f3d"
server_key_iv: "0c53c556a7ce680c818b481dba62d4e8"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:05.802510 13132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:05.816509 13132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:05.818821 13132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:05.818969 13132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:05.819204 13132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:05.819564 13132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:05.819600 13132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.819629 13132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:05.819646 13132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:05.835386 13132 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:43083
I20260923 22:14:05.835835 13132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:05.836560 13260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:43083 every 8 connection(s)
I20260923 22:14:05.840714 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 13132
I20260923 22:14:05.840826 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:14:05.841017 1554 external_mini_cluster.cc:1428] Setting key ba51492bacc1593bdb27a88dff960517
Sep 23 22:14:05 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:05.850073 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.837468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:38253 (local address 127.1.132.190:43623)
0923 22:14:05.837512 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.837515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.837853 (+ 338us) server_negotiation.cc:407] Connection header received
0923 22:14:05.838595 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.838596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.838609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.838640 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:05.839478 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.840310 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.843576 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.843908 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.843958 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.846866 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:05.846868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:05.846868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:05.846877 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:05.848511 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.849013 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.849014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.849014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.849071 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.849325 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.849326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.849326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.849462 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:05.849503 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.849892 (+ 389us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.849981 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:14:05.850979 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.837183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:43623 (local address 127.1.132.131:38253)
0923 22:14:05.837725 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.837770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.838444 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.838820 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.838824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.839067 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:05.839371 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.839377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.840484 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.840485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.843455 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.843460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.844057 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.844059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.844235 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.844992 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:05.845002 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:05.846728 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:05.848605 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.848612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.848615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.848928 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.849136 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:05.849137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:05.849137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:05.849261 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:05.849545 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:05.849551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:05.849725 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.850541 (+ 816us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.850722 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":78,"threads_started":1}
I20260923 22:14:05.851529 13261 heartbeater.cc:347] Connected to a master server at 127.1.132.190:43623
I20260923 22:14:05.851629 13261 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:05.851837 13261 heartbeater.cc:511] Master 127.1.132.190:43623 requested a full tablet report, sending...
I20260923 22:14:05.852593 12777 ts_manager.cc:194] Registered new tserver with Master: eccf426e0dab45a0842eacd7f542caa5 (127.1.132.131:43083)
I20260923 22:14:05.853120 12777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:38253
I20260923 22:14:05.853760 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:05.864787 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.856159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.1.132.190:43623)
0923 22:14:05.856229 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.856231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.856502 (+ 271us) server_negotiation.cc:407] Connection header received
0923 22:14:05.856621 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.856622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.856632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.856660 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:05.857486 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.858339 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.859573 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.859895 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.859937 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.861767 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:05.861769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:05.861769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:05.861777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:05.863445 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.863881 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.863883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.863883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.863953 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:05.864146 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:05.864146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:05.864147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:05.864313 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:05.864392 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.864599 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.864683 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:14:05.867295 12777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:14:05.869937 12777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:14:05.883867 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.874127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37959 (local address 127.0.0.1:48540)
0923 22:14:05.875655 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.875663 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.875700 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.876165 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.876169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.876171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:05.876295 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.876298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.880732 (+ 4434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.880733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.881917 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.881920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.883512 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.883513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.883619 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.883636 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.883703 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.883762 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1512,"thread_start_us":66,"threads_started":1}
I20260923 22:14:05.886984 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.874338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43083 (local address 127.0.0.1:57652)
0923 22:14:05.877332 (+ 2994us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.877342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.877399 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.877764 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.877765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.877767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:05.877814 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.877816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.879644 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.879645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.884673 (+ 5028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.884677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.886720 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.886721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.886786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.886791 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.886859 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.886906 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2976,"thread_start_us":72,"threads_started":1}
I20260923 22:14:05.887374 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.874783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.1.132.129:37959)
0923 22:14:05.875801 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.875806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.875817 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:14:05.875874 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.875876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.875893 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.875970 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:05.876386 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.880626 (+ 4240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.882534 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.883320 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.883394 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.883476 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.887225 (+ 3749us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.887285 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":67,"threads_started":1}
I20260923 22:14:05.887712 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.874943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.1.132.131:43083)
0923 22:14:05.876116 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.876120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.877461 (+ 1341us) server_negotiation.cc:407] Connection header received
0923 22:14:05.877505 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.877508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.877525 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.877595 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:05.877919 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.879030 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.884804 (+ 5774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.886544 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.886608 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.886686 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.887584 (+ 898us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.887632 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":73,"threads_started":1}
I20260923 22:14:05.888674 12923 tablet_service.cc:1467] Processing CreateTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 (DEFAULT_TABLE table=test-table [id=ab1feb08c41344aca6a2fad59a4320dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:05.889078 12923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8590c6d5f78242d097336971cdb3e0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:05.898087 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.873959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.1.132.130:38199)
0923 22:14:05.888299 (+ 14340us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:05.888304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:05.888315 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:14:05.888365 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:05.888367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:05.888384 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:05.888469 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:05.893762 (+ 5293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.894920 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.896378 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.897217 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.897268 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.897343 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:05.897917 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 22:14:05.897984 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14294,"thread_start_us":85,"threads_started":1}
I20260923 22:14:05.899644 13181 tablet_service.cc:1467] Processing CreateTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 (DEFAULT_TABLE table=test-table [id=ab1feb08c41344aca6a2fad59a4320dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:05.900074 13181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8590c6d5f78242d097336971cdb3e0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:05.898669 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:05.873847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:38199 (local address 127.0.0.1:36368)
0923 22:14:05.875431 (+ 1584us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:05.875480 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:05.875547 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:05.893590 (+ 18043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:05.893591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:05.893594 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:05.893653 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.893656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.895028 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.895029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.896258 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:05.896262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:05.897740 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:05.897741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:05.897809 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:05.897813 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:05.898519 (+ 706us) client_negotiation.cc:240] Negotiation successful
0923 22:14:05.898580 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1552,"thread_start_us":83,"threads_started":1}
I20260923 22:14:05.905575 13061 tablet_service.cc:1467] Processing CreateTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 (DEFAULT_TABLE table=test-table [id=ab1feb08c41344aca6a2fad59a4320dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:05.905972 13061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8590c6d5f78242d097336971cdb3e0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:05.909876 13280 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: Bootstrap starting.
I20260923 22:14:05.910110 13282 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Bootstrap starting.
I20260923 22:14:05.911103 13282 tablet_bootstrap.cc:654] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:05.912400 13281 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Bootstrap starting.
I20260923 22:14:05.912820 13280 tablet_bootstrap.cc:654] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:05.913883 13281 tablet_bootstrap.cc:654] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:05.914283 13282 log.cc:824] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:05.914709 13280 log.cc:824] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:05.915498 13281 log.cc:824] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:05.916431 13280 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: No bootstrap required, opened a new log
I20260923 22:14:05.916553 13280 ts_tablet_manager.cc:1397] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260923 22:14:05.918709 13280 raft_consensus.cc:348] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.919039 13280 raft_consensus.cc:374] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:05.919118 13280 raft_consensus.cc:729] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481db82ea94f4e3c81fb3807b82c68cc, State: Initialized, Role: FOLLOWER
I20260923 22:14:05.919346 13280 consensus_queue.cc:260] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.919924 13280 ts_tablet_manager.cc:1428] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:14:05.920083 12989 heartbeater.cc:503] Master 127.1.132.190:43623 was elected leader, sending a full tablet report...
I20260923 22:14:05.920859 13282 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: No bootstrap required, opened a new log
I20260923 22:14:05.920934 13282 ts_tablet_manager.cc:1397] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260923 22:14:05.922256 13281 tablet_bootstrap.cc:492] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: No bootstrap required, opened a new log
I20260923 22:14:05.922751 13281 ts_tablet_manager.cc:1397] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260923 22:14:05.923175 13282 raft_consensus.cc:348] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.923359 13282 raft_consensus.cc:374] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:05.923398 13282 raft_consensus.cc:729] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea1792640fb406e87d0242a66668927, State: Initialized, Role: FOLLOWER
I20260923 22:14:05.923518 13282 consensus_queue.cc:260] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.923774 13282 ts_tablet_manager.cc:1428] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:14:05.923890 13127 heartbeater.cc:503] Master 127.1.132.190:43623 was elected leader, sending a full tablet report...
I20260923 22:14:05.925622 13281 raft_consensus.cc:348] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.925895 13281 raft_consensus.cc:374] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:05.925966 13281 raft_consensus.cc:729] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eccf426e0dab45a0842eacd7f542caa5, State: Initialized, Role: FOLLOWER
I20260923 22:14:05.926138 13281 consensus_queue.cc:260] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:05.926446 13281 ts_tablet_manager.cc:1428] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:14:05.926605 13261 heartbeater.cc:503] Master 127.1.132.190:43623 was elected leader, sending a full tablet report...
Sep 23 22:14:06 dist-test-slave-l4q5 krb5kdc[12300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:06.076027 13286 raft_consensus.cc:482] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:06.076220 13286 raft_consensus.cc:504] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:06.076622 13286 leader_election.cc:288] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bea1792640fb406e87d0242a66668927 (127.1.132.130:38199), eccf426e0dab45a0842eacd7f542caa5 (127.1.132.131:43083)
I20260923 22:14:06.080687 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.076958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58121 (local address 127.1.132.130:38199)
0923 22:14:06.077004 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:06.077006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:06.077154 (+ 148us) server_negotiation.cc:407] Connection header received
0923 22:14:06.077264 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:06.077266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:06.077275 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:06.077332 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:06.077526 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.078256 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.079739 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.080312 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.080363 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:06.080373 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:06.080570 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:14:06.080609 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:06.080746 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.077053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:51265 (local address 127.1.132.131:43083)
0923 22:14:06.077089 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:06.077091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:06.077347 (+ 256us) server_negotiation.cc:407] Connection header received
0923 22:14:06.077450 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:06.077451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:06.077460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:06.077486 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:06.077681 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.078392 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.079761 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.080368 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.080416 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:06.080424 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:06.080630 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:14:06.080672 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:14:06.081004 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.076840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:43083 (local address 127.1.132.129:51265)
0923 22:14:06.077297 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:06.077306 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:06.077368 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:06.077548 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:06.077549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:06.077551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:06.077612 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:06.077615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.078518 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.078519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:06.079654 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:06.079657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.080482 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.080483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:06.080555 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:06.080559 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:06.080872 (+ 313us) client_negotiation.cc:240] Negotiation successful
0923 22:14:06.080920 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":97,"threads_started":1}
I20260923 22:14:06.081297 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.076788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:38199 (local address 127.1.132.129:58121)
0923 22:14:06.077068 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:06.077079 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:06.077187 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:06.077386 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:06.077388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:06.077390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:06.077458 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:06.077460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.078397 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.078397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:06.079606 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:06.079610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.080430 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.080431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:06.080498 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:06.080502 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:06.081166 (+ 664us) client_negotiation.cc:240] Negotiation successful
0923 22:14:06.081216 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":57,"threads_started":1}
I20260923 22:14:06.081362 13201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8590c6d5f78242d097336971cdb3e0a7" candidate_uuid: "481db82ea94f4e3c81fb3807b82c68cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eccf426e0dab45a0842eacd7f542caa5" is_pre_election: true
I20260923 22:14:06.081538 13201 raft_consensus.cc:2454] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 481db82ea94f4e3c81fb3807b82c68cc in term 0.
I20260923 22:14:06.081605 13064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8590c6d5f78242d097336971cdb3e0a7" candidate_uuid: "481db82ea94f4e3c81fb3807b82c68cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea1792640fb406e87d0242a66668927" is_pre_election: true
I20260923 22:14:06.081765 13064 raft_consensus.cc:2454] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 481db82ea94f4e3c81fb3807b82c68cc in term 0.
I20260923 22:14:06.081775 12878 leader_election.cc:302] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481db82ea94f4e3c81fb3807b82c68cc, eccf426e0dab45a0842eacd7f542caa5; no voters:
I20260923 22:14:06.081907 13286 raft_consensus.cc:2781] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:06.081980 13286 raft_consensus.cc:482] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:06.082005 13286 raft_consensus.cc:3037] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:06.082930 13286 raft_consensus.cc:504] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:06.083071 13286 leader_election.cc:288] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [CANDIDATE]: Term 1 election: Requested vote from peers bea1792640fb406e87d0242a66668927 (127.1.132.130:38199), eccf426e0dab45a0842eacd7f542caa5 (127.1.132.131:43083)
I20260923 22:14:06.083212 13064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8590c6d5f78242d097336971cdb3e0a7" candidate_uuid: "481db82ea94f4e3c81fb3807b82c68cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea1792640fb406e87d0242a66668927"
I20260923 22:14:06.083256 13201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8590c6d5f78242d097336971cdb3e0a7" candidate_uuid: "481db82ea94f4e3c81fb3807b82c68cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eccf426e0dab45a0842eacd7f542caa5"
I20260923 22:14:06.083285 13064 raft_consensus.cc:3037] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:06.083326 13201 raft_consensus.cc:3037] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:06.084215 13064 raft_consensus.cc:2454] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 481db82ea94f4e3c81fb3807b82c68cc in term 1.
I20260923 22:14:06.084234 13201 raft_consensus.cc:2454] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 481db82ea94f4e3c81fb3807b82c68cc in term 1.
I20260923 22:14:06.084391 12876 leader_election.cc:302] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481db82ea94f4e3c81fb3807b82c68cc, bea1792640fb406e87d0242a66668927; no voters:
I20260923 22:14:06.084553 13286 raft_consensus.cc:2781] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:06.084699 13286 raft_consensus.cc:686] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [term 1 LEADER]: Becoming Leader. State: Replica: 481db82ea94f4e3c81fb3807b82c68cc, State: Running, Role: LEADER
I20260923 22:14:06.084781 13286 consensus_queue.cc:237] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } }
I20260923 22:14:06.085713 12777 catalog_manager.cc:5495] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 481db82ea94f4e3c81fb3807b82c68cc (127.1.132.129). New cstate: current_term: 1 leader_uuid: "481db82ea94f4e3c81fb3807b82c68cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "481db82ea94f4e3c81fb3807b82c68cc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 } health_report { overall_health: UNKNOWN } } }
W20260923 22:14:06.087837 13262 tablet.cc:2369] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:14:06.106426 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.104014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.1.132.129:37959)
0923 22:14:06.104062 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:06.104064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:06.104072 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:06.104113 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:06.104114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:06.104136 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:06.104243 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:06.104366 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.104846 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.105647 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:06.105899 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:06.105954 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:06.106099 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:06.106210 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:06.106281 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:06.106301 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 22:14:06.106325 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:14:06.108470 13201 raft_consensus.cc:1261] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 1 FOLLOWER]: Refusing update from remote peer 481db82ea94f4e3c81fb3807b82c68cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:06.108484 13064 raft_consensus.cc:1261] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 1 FOLLOWER]: Refusing update from remote peer 481db82ea94f4e3c81fb3807b82c68cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:06.108839 13291 consensus_queue.cc:1035] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea1792640fb406e87d0242a66668927" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:06.108989 13286 consensus_queue.cc:1035] T 8590c6d5f78242d097336971cdb3e0a7 P 481db82ea94f4e3c81fb3807b82c68cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "eccf426e0dab45a0842eacd7f542caa5" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:06.111169 13294 mvcc.cc:205] Tried to move back new op lower bound from 7332665942457442304 to 7332665942363926528. Current Snapshot: MvccSnapshot[applied={T|T < 7332665942457442304}]
I20260923 22:14:06.111845 13295 mvcc.cc:205] Tried to move back new op lower bound from 7332665942457442304 to 7332665942363926528. Current Snapshot: MvccSnapshot[applied={T|T < 7332665942457442304}]
I20260923 22:14:06.112740 13296 mvcc.cc:205] Tried to move back new op lower bound from 7332665942457442304 to 7332665942363926528. Current Snapshot: MvccSnapshot[applied={T|T < 7332665942457442304}]
I20260923 22:14:06.114452 12777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:06.114545 12777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:06.115339 12777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86e4795f78944574baaf086fb1646e4e: Sending DeleteTablet for 3 replicas of tablet 8590c6d5f78242d097336971cdb3e0a7
I20260923 22:14:06.115657 13061 tablet_service.cc:1514] Processing DeleteTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:06 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36368
I20260923 22:14:06.115763 13181 tablet_service.cc:1514] Processing DeleteTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:06 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:57652
I20260923 22:14:06.115759 12923 tablet_service.cc:1514] Processing DeleteTablet for tablet 8590c6d5f78242d097336971cdb3e0a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:06 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:48540
I20260923 22:14:06.115859 13304 tablet_replica.cc:330] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: stopping tablet replica
I20260923 22:14:06.115940 13304 raft_consensus.cc:2229] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:06.116029 13304 raft_consensus.cc:2258] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:06.116608 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 12858
I20260923 22:14:06.117733 13305 tablet_replica.cc:330] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: stopping tablet replica
I20260923 22:14:06.117846 13305 raft_consensus.cc:2229] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:06.117960 13305 raft_consensus.cc:2258] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:06.118423 13305 ts_tablet_manager.cc:1905] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:06.118626 13304 ts_tablet_manager.cc:1905] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:14:06.120071 13304 ts_tablet_manager.cc:1918] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:06.120128 13304 log.cc:1196] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/8590c6d5f78242d097336971cdb3e0a7
I20260923 22:14:06.120355 13304 ts_tablet_manager.cc:1939] T 8590c6d5f78242d097336971cdb3e0a7 P bea1792640fb406e87d0242a66668927: Deleting consensus metadata
I20260923 22:14:06.120347 13305 ts_tablet_manager.cc:1918] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:14:06.120472 13305 log.cc:1196] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/8590c6d5f78242d097336971cdb3e0a7
I20260923 22:14:06.120777 12761 catalog_manager.cc:4841] TS bea1792640fb406e87d0242a66668927 (127.1.132.130:38199): tablet 8590c6d5f78242d097336971cdb3e0a7 (table test-table [id=ab1feb08c41344aca6a2fad59a4320dd]) successfully deleted
I20260923 22:14:06.121009 13305 ts_tablet_manager.cc:1939] T 8590c6d5f78242d097336971cdb3e0a7 P eccf426e0dab45a0842eacd7f542caa5: Deleting consensus metadata
I20260923 22:14:06.121380 12763 catalog_manager.cc:4841] TS eccf426e0dab45a0842eacd7f542caa5 (127.1.132.131:43083): tablet 8590c6d5f78242d097336971cdb3e0a7 (table test-table [id=ab1feb08c41344aca6a2fad59a4320dd]) successfully deleted
W20260923 22:14:06.126910 12764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:37959 (error 108)
I20260923 22:14:06.127053 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 12997
I20260923 22:14:06.128342 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:06.128218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37959 (local address 127.0.0.1:48562)
0923 22:14:06.128269 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:06.128303 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:14:06.128453 12764 catalog_manager.cc:4584] TS 481db82ea94f4e3c81fb3807b82c68cc (127.1.132.129:37959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8590c6d5f78242d097336971cdb3e0a7: Network error: Client connection negotiation failed: client connection to 127.1.132.129:37959: connect: Connection refused (error 111)
I20260923 22:14:06.131667 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 13132
I20260923 22:14:06.135428 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 12745
I20260923 22:14:06.140046 1554 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:14:06.475443 1554 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:14:06.475553 1554 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:14:06.811154 1554 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:14:06.811 UTC [12330] LOG: received smart shutdown request
2026-09-23 22:14:06.815 UTC [12330] LOG: background worker "logical replication launcher" (PID 12339) exited with exit code 1
2026-09-23 22:14:06.815 UTC [12334] LOG: shutting down
2026-09-23 22:14:07.025 UTC [12330] LOG: database system is shut down
2026-09-23T22:14:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47609 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:07 dist-test-slave-l4q5 krb5kdc[13316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:07 dist-test-slave-l4q5 krb5kdc[13316](info): set up 2 sockets
Sep 23 22:14:07 dist-test-slave-l4q5 krb5kdc[13316](info): commencing operation
krb5kdc: starting...
W20260923 22:14:08.075958 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:14:08 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangeradmin_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangeradmin_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangerlookup_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangerlookup_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
I20260923 22:14:08.251775 1554 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1S3mcj/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/postgres -l logfile start
2026-09-23 22:14:11.151 UTC [13346] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:14:11.151 UTC [13346] LOG: listening on IPv4 address "127.1.132.148", port 43471
2026-09-23 22:14:11.156 UTC [13346] LOG: listening on Unix socket "/tmp/.s.PGSQL.43471"
2026-09-23 22:14:11.188 UTC [13349] LOG: database system was shut down at 2026-09-23 22:14:09 UTC
2026-09-23 22:14:11.193 UTC [13346] LOG: database system is ready to accept connections
I20260923 22:14:12.051829 1554 mini_postgres.cc:96] Postgres bound to 43471
2026-09-23 22:14:12.057 UTC [13357] FATAL: database "slave" does not exist
127.1.132.148:43471 - accepting connections
I20260923 22:14:12.058140 1554 mini_ranger.cc:161] Starting Ranger...
I20260923 22:14:12.072425 1554 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:14:12.633208 1554 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:14:12.633293 1554 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin
2026-09-23 22:14:12,910 [I] DB FLAVOR :POSTGRES
2026-09-23 22:14:12,910 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:14:12,910 [I] Checking connection..
2026-09-23 22:14:12,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:14:13,159 [I] Checking connection passed.
2026-09-23 22:14:13,159 [I] --------- Verifying version history table ---------
2026-09-23 22:14:13,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:13,412 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:14:13,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:13,652 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:14:13,652 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:14:13,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:14:13.887 UTC [13421] WARNING: there is no transaction in progress
2026-09-23 22:14:13,893 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:14:13,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:14:14,146 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:14:14,146 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:14:14,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:14:14,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:14:14,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-l4q5.c.gcp-upstream.internal','N') ;"
2026-09-23 22:14:14,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:14:15,140 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:14:15,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:14:15,405 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:14:15,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:14:15,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:14:15,919 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:14:15,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:14:17.002 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.026 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.047 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.069 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.089 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.109 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.180 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.191 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.212 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.223 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.235 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.251 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.262 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.270 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.278 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:17.286 UTC [13529] WARNING: there is no transaction in progress
2026-09-23 22:14:18,071 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:14:18,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:14:18,320 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:14:18,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:14:18,559 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:14:18,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:14:18,813 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:14:18,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:14:19,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:14:19,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-l4q5.c.gcp-upstream.internal';"
2026-09-23 22:14:19,566 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:14:19,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:14:19,819 [I] DB_PATCHES have already been applied
I20260923 22:14:19.825050 1554 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1S3mcj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:14:19.825142 1554 mini_ranger.cc:193] Using host: 127.1.132.148
I20260923 22:14:19.828114 1554 mini_ranger.cc:227] Ranger admin URL: http://127.1.132.148:54639
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:14:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:14:20.841998 1554 mini_ranger.cc:160] Time spent starting Ranger: real 8.784s user 0.001s sys 0.007s
Sep 23, 2026 10:14:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:14:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:14:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:14:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:14:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:14:42.432489 1554 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangerkms_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/rangerkms_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/HTTP_127.1.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:14:42 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:14:42.515925 1554 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:14:42.524672 1554 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:14:43.564718 1554 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:14:43.564785 1554 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms
I20260923 22:14:43.564796 1554 mini_ranger_kms.cc:227] Using postgres at 127.1.132.148:43471
2026-09-23 22:14:43,713 [I] DB FLAVOR :POSTGRES
2026-09-23 22:14:43,713 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:14:43,713 [I] Checking connection
2026-09-23 22:14:43,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:14:43,945 [I] connection success
2026-09-23 22:14:43,945 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:14:43,946 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:14:43,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:14:44,205 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:14:44,205 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:14:44,205 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:14:44,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.132.148:43471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:14:44,479 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:14:44.993839 1554 mini_ranger_kms.cc:310] Created kms service
I20260923 22:14:45.194272 1554 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:14:45.242107 1554 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:14:45 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201682, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:45.666489 1554 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:14:45.666692 1554 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms:/tmp/dist-test-task1S3mcj/build/release/bin/postgresql.jar:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1S3mcj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:14:45.666726 1554 mini_ranger_kms.cc:240] Using host: 127.1.132.148
I20260923 22:14:45.669948 1554 mini_ranger_kms.cc:276] Ranger KMS PID: 13709
I20260923 22:14:45.670033 1554 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.132.148:47473
Sep 23, 2026 10:14:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:14:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:14:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:14:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:14:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:14:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:14:47.709862 1554 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.194s user 0.006s sys 0.124s
I20260923 22:14:47.709945 1554 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:14:47.709990 1554 mini_ranger_kms.cc:401] 127.1.132.148:47473/kms/v1/keys
Sep 23 22:14:47 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:14:48 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
Sep 23 22:14:48 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.132.148@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
Sep 23 22:14:51 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:14:51.052366 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35105
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35105
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:51.132174 13760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:51.132447 13760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:51.132476 13760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:51.133714 13760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:51.133767 13760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:51.133786 13760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:51.133801 13760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:51.133817 13760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:51.135516 13760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.132.148:47473/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35105
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:51.135918 13760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:51.136162 13760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:51.141472 13765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:51.141551 13760 server_base.cc:1034] running on GCE node
W20260923 22:14:51.141472 13766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:51.141661 13768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:51.141927 13760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:51.142228 13760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:51.143388 13760 hybrid_clock.cc:648] HybridClock initialized: now 1790201691143374 us; error 33 us; skew 500 ppm
Sep 23 22:14:51 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:51.149657 13760 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:14:51.150246 13760 webserver.cc:460] Webserver started at http://127.1.132.190:36297/ using document root <none> and password file <none>
I20260923 22:14:51.150522 13760 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:51.150576 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:51.150713 13760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:51 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:51.203014 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "5833959ae41f44d786c60a4f8833823b"
format_stamp: "Formatted at 2026-09-23 22:14:51 on dist-test-slave-l4q5"
server_key: "f03d68cdefa5f5f040a14a2998748d7a"
server_key_iv: "fb7732cc3a3e0d2fcf40b0e035534d0b"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:51.203392 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "5833959ae41f44d786c60a4f8833823b"
format_stamp: "Formatted at 2026-09-23 22:14:51 on dist-test-slave-l4q5"
server_key: "f03d68cdefa5f5f040a14a2998748d7a"
server_key_iv: "fb7732cc3a3e0d2fcf40b0e035534d0b"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:51.205075 13760 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:51.220359 13775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:51.220664 13760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:51.220758 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "5833959ae41f44d786c60a4f8833823b"
format_stamp: "Formatted at 2026-09-23 22:14:51 on dist-test-slave-l4q5"
server_key: "f03d68cdefa5f5f040a14a2998748d7a"
server_key_iv: "fb7732cc3a3e0d2fcf40b0e035534d0b"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:51.220846 13760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:51.256366 13760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:51.258916 13760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:51.259066 13760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:51.264071 13760 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35105
I20260923 22:14:51.264132 13827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35105 every 8 connection(s)
I20260923 22:14:51.264616 13760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:51.265311 13828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:51.267616 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Bootstrap starting.
I20260923 22:14:51.268213 13828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:51.268479 13828 log.cc:824] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:51.269198 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: No bootstrap required, opened a new log
I20260923 22:14:51.271237 13828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:51.271401 13828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:51.271433 13828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5833959ae41f44d786c60a4f8833823b, State: Initialized, Role: FOLLOWER
I20260923 22:14:51.271575 13828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:51.271651 13828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:51.271682 13828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:51.271739 13828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:51.272389 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 13760
I20260923 22:14:51.272435 13828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:51.272486 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:14:51.272579 13828 leader_election.cc:302] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5833959ae41f44d786c60a4f8833823b; no voters:
I20260923 22:14:51.272657 1554 external_mini_cluster.cc:1428] Setting key da1742e7c58fdfda6a8b6003b25ea750
I20260923 22:14:51.272784 13828 leader_election.cc:288] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:51.272835 13831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:51.273083 13831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 LEADER]: Becoming Leader. State: Replica: 5833959ae41f44d786c60a4f8833823b, State: Running, Role: LEADER
I20260923 22:14:51.273255 13831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:51.273464 13828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:51.273665 13832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5833959ae41f44d786c60a4f8833823b. Latest consensus state: current_term: 1 leader_uuid: "5833959ae41f44d786c60a4f8833823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } } }
I20260923 22:14:51.273739 13832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:51.273842 13832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5833959ae41f44d786c60a4f8833823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } } }
I20260923 22:14:51.273877 13832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:51.274343 13837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:51.274856 13837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:51.275666 13828 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 23 22:14:51 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:51.287587 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:51.273303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38566 (local address 127.1.132.190:35105)
0923 22:14:51.273478 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:51.273528 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:51.274973 (+ 1445us) server_negotiation.cc:407] Connection header received
0923 22:14:51.275139 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:51.275146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:51.275201 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:51.275319 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:51.276039 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:51.276990 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:51.277743 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:51.280628 (+ 2885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:51.280819 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:51.283545 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:51.283550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:51.283550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:51.283564 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:51.285558 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:51.285980 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:51.285984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:51.285984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:51.286104 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:51.286302 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:51.286303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:51.286303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:51.286619 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:51.286747 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:51.287196 (+ 449us) server_negotiation.cc:299] Negotiation successful
0923 22:14:51.287335 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":51,"threads_started":1}
I20260923 22:14:51.288725 13837 catalog_manager.cc:1347] Generated new cluster ID: c5923aa00d6f4afdbb9b311c005e78f9
I20260923 22:14:51.288781 13837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:51.304255 13837 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:51.305096 13837 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:51.321177 13837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Generated new TSK 0
I20260923 22:14:51.321507 13837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:14:52.016374 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.131338 13873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.131636 13873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.131685 13873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.132982 13873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.133029 13873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.133065 13873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:14:52.134719 13873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:52.134977 13873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.135280 13873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.140393 13881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.140398 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.140631 13873 server_base.cc:1034] running on GCE node
W20260923 22:14:52.140746 13879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.141031 13873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.141284 13873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.142428 13873 hybrid_clock.cc:648] HybridClock initialized: now 1790201692142409 us; error 34 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.145764 13873 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:14:52.146306 13873 webserver.cc:460] Webserver started at http://127.1.132.129:44999/ using document root <none> and password file <none>
I20260923 22:14:52.146586 13873 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.146649 13873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.146834 13873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:52.161121 13873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "bd90daf3b54641a48d8e3fb34e26a265"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "e4680cea08e76ed9eaffc2dc98ae83d5"
server_key_iv: "0be5dd8f53b3624476b15740f07727ce"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.161787 13873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "bd90daf3b54641a48d8e3fb34e26a265"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "e4680cea08e76ed9eaffc2dc98ae83d5"
server_key_iv: "0be5dd8f53b3624476b15740f07727ce"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.168493 13873 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260923 22:14:52.190316 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.190727 13873 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260923 22:14:52.190868 13873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "bd90daf3b54641a48d8e3fb34e26a265"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "e4680cea08e76ed9eaffc2dc98ae83d5"
server_key_iv: "0be5dd8f53b3624476b15740f07727ce"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.190999 13873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.209672 13873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.212149 13873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.212390 13873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.212711 13873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.213132 13873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.213214 13873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.213303 13873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.213372 13873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.219466 13873 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:45723
I20260923 22:14:52.220006 13873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:52.222788 14003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:45723 every 8 connection(s)
I20260923 22:14:52.226497 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 13873
I20260923 22:14:52.226605 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:14:52.226784 1554 external_mini_cluster.cc:1428] Setting key ce4226c022cd44f3c0d5e8f6b284a9ff
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:52.236258 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.223473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.129:40599)
0923 22:14:52.224290 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.224337 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.225096 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.225505 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.225509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.225917 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.226214 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.226220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.227896 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.227897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.229143 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.229146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.229669 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.229670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.229810 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.230397 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.230408 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.232036 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.234421 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.234431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.234434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.234840 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.235055 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.235055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.235056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.235193 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.235490 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.235496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.235673 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.235806 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.235983 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":91,"threads_started":1}
I20260923 22:14:52.236828 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.223770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:40599 (local address 127.1.132.190:35105)
0923 22:14:52.225278 (+ 1508us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.225280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.225288 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:52.225313 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.225314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.225326 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.225383 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.226314 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.227805 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.229240 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.229533 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.229595 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.232145 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.232146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.232146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.232154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.234312 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.234923 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.234925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.234925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.234992 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.235263 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.235264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.235264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.235404 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.235463 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.236633 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.236738 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1478,"thread_start_us":69,"threads_started":1}
I20260923 22:14:52.237593 14004 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:52.237691 14004 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.237900 14004 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:52.238531 13792 ts_manager.cc:194] Registered new tserver with Master: bd90daf3b54641a48d8e3fb34e26a265 (127.1.132.129:45723)
I20260923 22:14:52.239408 13792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:40599
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:14:52.270767 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:14:52.389424 14012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.389710 14012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.389757 14012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.391561 14012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.391654 14012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.391710 14012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:14:52.393954 14012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:52.394222 14012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.394515 14012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.400681 14018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.400786 14019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.400686 14021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.401856 14012 server_base.cc:1034] running on GCE node
I20260923 22:14:52.402045 14012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.402279 14012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.403436 14012 hybrid_clock.cc:648] HybridClock initialized: now 1790201692403420 us; error 32 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.406162 14012 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:14:52.406735 14012 webserver.cc:460] Webserver started at http://127.1.132.130:43489/ using document root <none> and password file <none>
I20260923 22:14:52.407089 14012 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.407174 14012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.407361 14012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:52.421012 14012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "a35a6377503d41c6aab1f698008f0ada"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "3e84804dff788ea1ea5f6866da34f681"
server_key_iv: "0e3a22c974e25447d358e6f2a9e4f6da"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.421464 14012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "a35a6377503d41c6aab1f698008f0ada"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "3e84804dff788ea1ea5f6866da34f681"
server_key_iv: "0e3a22c974e25447d358e6f2a9e4f6da"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.423676 14012 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 22:14:52.444054 14028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.444371 14012 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260923 22:14:52.444504 14012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "a35a6377503d41c6aab1f698008f0ada"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "3e84804dff788ea1ea5f6866da34f681"
server_key_iv: "0e3a22c974e25447d358e6f2a9e4f6da"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.444628 14012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.459859 14012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.462414 14012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.462633 14012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.462944 14012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.463395 14012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.463486 14012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.463563 14012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.463619 14012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.471480 14012 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:44233
I20260923 22:14:52.471640 14141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:44233 every 8 connection(s)
I20260923 22:14:52.472052 14012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:52.481809 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14012
I20260923 22:14:52.481901 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:14:52.482095 1554 external_mini_cluster.cc:1428] Setting key 14aeaa67d552a48bc075424cf01edcab
I20260923 22:14:52.485898 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.473852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:41585 (local address 127.1.132.190:35105)
0923 22:14:52.473907 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.473909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.474149 (+ 240us) server_negotiation.cc:407] Connection header received
0923 22:14:52.474910 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.474912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.474929 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.475016 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.475778 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.476600 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.478083 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.478400 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.478472 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.481095 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.481097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.481098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.481106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.482745 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.484786 (+ 2041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.484788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.484789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.484855 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.485147 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.485148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.485149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.485287 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.485329 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.485712 (+ 383us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.485806 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:14:52.488381 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.473569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.130:41585)
0923 22:14:52.474008 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.474056 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.474786 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.475103 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.475108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.475365 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.475663 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.475670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.476710 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.476711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.477989 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.477992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.478525 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.478526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.478681 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.479298 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.479308 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.480987 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.484337 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.484346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.484348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.484683 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.484923 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.484924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.484925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.485075 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.485372 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.485378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.485548 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.485670 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.485834 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":74,"threads_started":1}
I20260923 22:14:52.488950 14142 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:52.489045 14142 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.489254 14142 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:52.489669 13792 ts_manager.cc:194] Registered new tserver with Master: a35a6377503d41c6aab1f698008f0ada (127.1.132.130:44233)
I20260923 22:14:52.490171 13792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:41585
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:14:52.513487 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.619292 14147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.619668 14147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.619721 14147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.621702 14147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.621827 14147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.621896 14147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:14:52.623955 14147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:52.624269 14147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.624511 14147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.629558 14155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.629580 14152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.630290 14153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.630887 14147 server_base.cc:1034] running on GCE node
I20260923 22:14:52.631089 14147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.631366 14147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.632509 14147 hybrid_clock.cc:648] HybridClock initialized: now 1790201692632488 us; error 35 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.634917 14147 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:14:52.635352 14147 webserver.cc:460] Webserver started at http://127.1.132.131:36295/ using document root <none> and password file <none>
I20260923 22:14:52.635608 14147 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.635672 14147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:52.635795 14147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:52.659093 14147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "9b23b4b59bcf4de59324bee5621357b3"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "bf98a73ac25b23ccf550170952142c3d"
server_key_iv: "20e66290fcb2a8217bed4a5b70e2d1fc"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.659677 14147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "9b23b4b59bcf4de59324bee5621357b3"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "bf98a73ac25b23ccf550170952142c3d"
server_key_iv: "20e66290fcb2a8217bed4a5b70e2d1fc"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.668659 14147 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260923 22:14:52.704372 14162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.704674 14147 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.001s
I20260923 22:14:52.704784 14147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9b23b4b59bcf4de59324bee5621357b3"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "bf98a73ac25b23ccf550170952142c3d"
server_key_iv: "20e66290fcb2a8217bed4a5b70e2d1fc"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:52.704907 14147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:52.724992 14147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:52.727492 14147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:52.727710 14147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:52.727981 14147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:52.728442 14147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:52.728531 14147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.728600 14147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:52.728652 14147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:52.734834 14147 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45331
I20260923 22:14:52.735363 14147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:52.741257 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14147
I20260923 22:14:52.741377 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:14:52.741551 1554 external_mini_cluster.cc:1428] Setting key 95b28d10e87109e6df7a3d23783e0617
I20260923 22:14:52.743054 14275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45331 every 8 connection(s)
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:52.769630 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.744035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:35891 (local address 127.1.132.190:35105)
0923 22:14:52.744078 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:52.744080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:52.753635 (+ 9555us) server_negotiation.cc:407] Connection header received
0923 22:14:52.754373 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:52.754374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:52.754386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:52.754414 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:52.755281 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.756065 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.757644 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.758005 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.758050 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.766233 (+ 8183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:52.766235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:52.766235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:52.766245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:52.767912 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.768498 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.768500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.768501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.768562 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:52.768863 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:52.768864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:52.768865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:52.769003 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:52.769050 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:52.769451 (+ 401us) server_negotiation.cc:299] Negotiation successful
0923 22:14:52.769535 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:52.770841 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:52.743767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.131:35891)
0923 22:14:52.753470 (+ 9703us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:52.753550 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:52.754250 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:52.754617 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:52.754621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:52.754888 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:52.755174 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.755180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.756210 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.756211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.757533 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:52.757537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:52.758163 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:52.758164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:52.758314 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:52.764308 (+ 5994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:52.764330 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:52.766099 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:52.768026 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.768038 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.768041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.768404 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.768627 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:52.768628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:52.768628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:52.768787 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:52.769083 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:52.769090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:52.769274 (+ 184us) client_negotiation.cc:769] Sending connection context
0923 22:14:52.770351 (+ 1077us) client_negotiation.cc:240] Negotiation successful
0923 22:14:52.770586 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9633,"thread_start_us":85,"threads_started":1}
I20260923 22:14:52.771508 14276 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:52.771610 14276 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:52.771823 14276 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:52.772198 13792 ts_manager.cc:194] Registered new tserver with Master: 9b23b4b59bcf4de59324bee5621357b3 (127.1.132.131:45331)
I20260923 22:14:52.772693 13792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:35891
I20260923 22:14:52.774849 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:52.775525 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 13873
I20260923 22:14:52.797159 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14012
I20260923 22:14:52.803220 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14147
I20260923 22:14:52.810674 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 13760
I20260923 22:14:52.817303 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35105
--webserver_interface=127.1.132.190
--webserver_port=36297
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:35105
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:52.906544 14279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:52.906764 14279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:52.906787 14279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:52.908018 14279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:52.908077 14279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:52.908089 14279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:52.908102 14279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:52.908111 14279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:52.909788 14279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:35105
--ranger_config_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=36297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:52.910017 14279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:52.910234 14279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:52.915433 14287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.915433 14284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:52.915716 14285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:52.916065 14279 server_base.cc:1034] running on GCE node
I20260923 22:14:52.916337 14279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:52.916656 14279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:52.917876 14279 hybrid_clock.cc:648] HybridClock initialized: now 1790201692917818 us; error 66 us; skew 500 ppm
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:52.920969 14279 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:14:52.921464 14279 webserver.cc:460] Webserver started at http://127.1.132.190:36297/ using document root <none> and password file <none>
I20260923 22:14:52.921713 14279 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:52.921768 14279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:52 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201692, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:53.003541 14279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.004251 14295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.004426 14279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:53.004546 14279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "5833959ae41f44d786c60a4f8833823b"
format_stamp: "Formatted at 2026-09-23 22:14:51 on dist-test-slave-l4q5"
server_key: "f03d68cdefa5f5f040a14a2998748d7a"
server_key_iv: "fb7732cc3a3e0d2fcf40b0e035534d0b"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:53.004944 14279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.017462 14279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.019345 14279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.019483 14279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.023897 14279 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:35105
I20260923 22:14:53.023960 14347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:35105 every 8 connection(s)
I20260923 22:14:53.024349 14279 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:53.024928 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14279
I20260923 22:14:53.025236 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:45723
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:35105
--webserver_port=44999
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.025672 14348 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:14:53.026265 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Bootstrap starting.
I20260923 22:14:53.036993 14348 log.cc:824] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:53.039240 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:14:53.039565 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Bootstrap complete.
I20260923 22:14:53.042785 14348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:53.043104 14348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5833959ae41f44d786c60a4f8833823b, State: Initialized, Role: FOLLOWER
I20260923 22:14:53.043299 14348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:53.043391 14348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:53.043447 14348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:53.043514 14348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:14:53.044986 14348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:53.045102 14348 leader_election.cc:302] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5833959ae41f44d786c60a4f8833823b; no voters:
I20260923 22:14:53.045313 14348 leader_election.cc:288] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:14:53.045421 14352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:14:53.045593 14348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:53.045610 14352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [term 2 LEADER]: Becoming Leader. State: Replica: 5833959ae41f44d786c60a4f8833823b, State: Running, Role: LEADER
I20260923 22:14:53.045848 14352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } }
I20260923 22:14:53.046108 14352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5833959ae41f44d786c60a4f8833823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } } }
I20260923 22:14:53.046162 14352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:53.046257 14352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5833959ae41f44d786c60a4f8833823b. Latest consensus state: current_term: 2 leader_uuid: "5833959ae41f44d786c60a4f8833823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5833959ae41f44d786c60a4f8833823b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 35105 } } }
I20260923 22:14:53.046288 14352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:53.046494 14360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:53.047006 14360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:53.047215 14360 catalog_manager.cc:1259] Loaded cluster ID: c5923aa00d6f4afdbb9b311c005e78f9
I20260923 22:14:53.047288 14360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:53.049045 14360 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:53.049335 14360 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Loaded TSK: 0
I20260923 22:14:53.049598 14360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:14:53.050400 14348 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260923 22:14:53.142925 14350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.143314 14350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.143391 14350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.145325 14350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.145556 14350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.145649 14350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:14:53.147876 14350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:45723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=44999
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:53.148376 14350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.148698 14350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.155320 14386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.155310 14385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.156395 14350 server_base.cc:1034] running on GCE node
W20260923 22:14:53.156605 14388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.156873 14350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.157159 14350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.170531 14350 hybrid_clock.cc:648] HybridClock initialized: now 1790201693170487 us; error 59 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.173894 14350 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:14:53.174492 14350 webserver.cc:460] Webserver started at http://127.1.132.129:44999/ using document root <none> and password file <none>
I20260923 22:14:53.174801 14350 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.174937 14350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:53.194443 14350 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:53.195611 14396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.195825 14350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:53.195914 14350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "bd90daf3b54641a48d8e3fb34e26a265"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "e4680cea08e76ed9eaffc2dc98ae83d5"
server_key_iv: "0be5dd8f53b3624476b15740f07727ce"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:53.196424 14350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.217216 14350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.222553 14350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.222792 14350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.223052 14350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.223654 14350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.223743 14350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.223840 14350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.223908 14350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.230203 14350 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:45723
I20260923 22:14:53.230822 14350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:14:53.231737 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14350
I20260923 22:14:53.232003 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:44233
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:35105
--webserver_port=43489
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.235116 14509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:45723 every 8 connection(s)
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:53.261180 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.236005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.129:45525)
0923 22:14:53.239510 (+ 3505us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.239573 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.241761 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.242597 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.242601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.242855 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.243136 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.243143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.246240 (+ 3097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.246242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.252709 (+ 6467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.252717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.253807 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.253808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.253952 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.254570 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.254581 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.256221 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.259130 (+ 2909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.259137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.259140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.259432 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.259810 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.259811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.259812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.259945 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.260487 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.260493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.260660 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.260762 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.260934 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3448,"thread_start_us":117,"threads_started":1}
I20260923 22:14:53.262094 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.236438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45525 (local address 127.1.132.190:35105)
0923 22:14:53.240309 (+ 3871us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.240319 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.240369 (+ 50us) server_negotiation.cc:407] Connection header received
0923 22:14:53.242120 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.242128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.242202 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.242353 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.243271 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.246013 (+ 2742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.252858 (+ 6845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.253536 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.253650 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.256360 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.256366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.256366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.256382 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.259023 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.259533 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.259537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.259538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.259731 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.260021 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.260022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.260023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.260381 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.260457 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.261665 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.261888 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3802,"thread_start_us":73,"threads_started":1}
I20260923 22:14:53.263221 14510 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:53.263334 14510 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.263533 14510 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.264487 14510 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:35105 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:53.264566 14510 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.264632 14510 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:53.264979 14510 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.265060 14510 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.265352 14510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:14:53.362782 14512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.363075 14512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.363111 14512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.364967 14512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.365123 14512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.365208 14512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:14:53.367089 14512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:44233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=43489
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:53.367370 14512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.367661 14512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.372872 14520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.373230 14519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.373381 14512 server_base.cc:1034] running on GCE node
W20260923 22:14:53.373761 14522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.374049 14512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.374326 14512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.375492 14512 hybrid_clock.cc:648] HybridClock initialized: now 1790201693375477 us; error 33 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.377990 14512 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:14:53.378463 14512 webserver.cc:460] Webserver started at http://127.1.132.130:43489/ using document root <none> and password file <none>
I20260923 22:14:53.378727 14512 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.378844 14512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:53.393110 14512 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 22:14:53.394258 14530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.394438 14512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:14:53.394536 14512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "a35a6377503d41c6aab1f698008f0ada"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "3e84804dff788ea1ea5f6866da34f681"
server_key_iv: "0e3a22c974e25447d358e6f2a9e4f6da"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:53.395020 14512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.417512 14512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.419767 14512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.419917 14512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.420161 14512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.420544 14512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.420583 14512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.420624 14512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.420670 14512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.426015 14512 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:44233
I20260923 22:14:53.426232 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:44233 every 8 connection(s)
I20260923 22:14:53.426518 14512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:53.432402 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14512
I20260923 22:14:53.432647 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.132.148:47473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:45331
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:35105
--webserver_port=36295
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:53.437408 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.427353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:40011 (local address 127.1.132.190:35105)
0923 22:14:53.427396 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.427398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.427688 (+ 290us) server_negotiation.cc:407] Connection header received
0923 22:14:53.428156 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.428158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.428175 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.428254 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.428803 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.429301 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.430337 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.430547 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.430604 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.434238 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.434240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.434241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.434250 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.435663 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.436235 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.436238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.436238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.436302 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.436609 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.436610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.436611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.436768 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.436814 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.437212 (+ 398us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.437309 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:14:53.437989 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.427370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.130:40011)
0923 22:14:53.427538 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.427576 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.428054 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.428327 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.428330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.428500 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.428709 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.428714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.429390 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.429391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.430245 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.430249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.430638 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.430639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.430752 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.431292 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.431300 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.434121 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.435771 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.435781 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.435785 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.436108 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.436375 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.436376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.436376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.436533 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.436864 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.436871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.437052 (+ 181us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.437555 (+ 503us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.437749 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260923 22:14:53.438644 14644 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:53.438750 14644 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.438938 14644 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.439628 14644 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:35105 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:53.439708 14644 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.439775 14644 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:53.440160 14644 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.440284 14644 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.440623 14644 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:14:53.549544 14647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:53.549942 14647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:53.550041 14647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:53.551986 14647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:53.552110 14647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:53.552215 14647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:14:53.554453 14647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.132.148:47473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:45331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=36295
--tserver_master_addrs=127.1.132.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:53.554716 14647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:53.555025 14647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:53.561475 14653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:53.561749 14655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.562004 14647 server_base.cc:1034] running on GCE node
W20260923 22:14:53.562182 14652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:53.562433 14647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:53.562743 14647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:53.563926 14647 hybrid_clock.cc:648] HybridClock initialized: now 1790201693563907 us; error 37 us; skew 500 ppm
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:53.567348 14647 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:14:53.567947 14647 webserver.cc:460] Webserver started at http://127.1.132.131:36295/ using document root <none> and password file <none>
I20260923 22:14:53.568291 14647 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:53.568377 14647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:53.584381 14647 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 22:14:53.585529 14663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.585788 14647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:53.585896 14647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9b23b4b59bcf4de59324bee5621357b3"
format_stamp: "Formatted at 2026-09-23 22:14:52 on dist-test-slave-l4q5"
server_key: "bf98a73ac25b23ccf550170952142c3d"
server_key_iv: "20e66290fcb2a8217bed4a5b70e2d1fc"
server_key_version: "kuduclusterkey@0"
I20260923 22:14:53.586369 14647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:53.603147 14647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:53.606393 14647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:53.606622 14647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:53.606925 14647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:14:53.607365 14647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:14:53.607455 14647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.607520 14647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:14:53.607565 14647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:53.614140 14647 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45331
I20260923 22:14:53.614334 14777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45331 every 8 connection(s)
I20260923 22:14:53.614639 14647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:14:53.621709 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14647
Sep 23 22:14:53 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201693, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:53.630574 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.615384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:55977 (local address 127.1.132.190:35105)
0923 22:14:53.615429 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.615431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.615681 (+ 250us) server_negotiation.cc:407] Connection header received
0923 22:14:53.616364 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.616366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.616379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.616548 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.617345 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.618134 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.619670 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.619952 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.620035 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.627467 (+ 7432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.627469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.627469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.627477 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.628930 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.629391 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.629393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.629393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.629454 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.629787 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.629788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.629789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.629938 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.630014 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.630374 (+ 360us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.630461 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:53.630753 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.615371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:35105 (local address 127.1.132.131:55977)
0923 22:14:53.615529 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:53.615578 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:53.616254 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:53.616682 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:53.616687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:53.616932 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:53.617233 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.617240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.618251 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.618252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.619552 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:53.619556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.620076 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.620078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:53.621551 (+ 1473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.622521 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:53.622532 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:53.627345 (+ 4813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:53.629035 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.629046 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.629049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.629302 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.629539 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:53.629540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:53.629540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:53.629709 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:53.630016 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:53.630024 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:53.630218 (+ 194us) client_negotiation.cc:769] Sending connection context
0923 22:14:53.630337 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 22:14:53.630504 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260923 22:14:53.631646 14778 heartbeater.cc:347] Connected to a master server at 127.1.132.190:35105
I20260923 22:14:53.631791 14778 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.632036 14778 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.632740 14778 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:35105 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:14:53.632956 14778 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.633060 14778 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:53.633369 14778 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:53.633474 14778 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
W20260923 22:14:53.633771 14778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:14:53.633846 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:53.622160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.1.132.190:35105)
0923 22:14:53.623228 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:53.623230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:53.623432 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:14:53.623456 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:53.623457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:53.623470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:53.623522 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:53.624297 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.625055 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.626183 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:53.626478 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:53.626545 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:53.629038 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:53.629039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:53.629040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:53.629047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:53.632067 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.632567 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.632568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.632568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.632635 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:53.632861 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:53.632862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:53.632862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:53.633007 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:53.633190 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:53.633646 (+ 456us) server_negotiation.cc:299] Negotiation successful
0923 22:14:53.633740 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":78,"threads_started":1}
Sep 23 22:14:54 dist-test-slave-l4q5 krb5kdc[13316](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:54.265514 14510 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:54.265686 14510 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:54.266137 14311 ts_manager.cc:194] Registered new tserver with Master: bd90daf3b54641a48d8e3fb34e26a265 (127.1.132.129:45723)
I20260923 22:14:54.267155 14311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:45525
I20260923 22:14:54.440821 14644 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:54.440969 14644 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:54.441304 14311 ts_manager.cc:194] Registered new tserver with Master: a35a6377503d41c6aab1f698008f0ada (127.1.132.130:44233)
I20260923 22:14:54.441799 14311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:40011
Sep 23 22:14:54 dist-test-slave-l4q5 krb5kdc[13316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201694, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for HTTP/127.1.132.148@KRBTEST.COM
I20260923 22:14:54.633977 14778 heartbeater.cc:464] Registering TS with master...
I20260923 22:14:54.634164 14778 heartbeater.cc:511] Master 127.1.132.190:35105 requested a full tablet report, sending...
I20260923 22:14:54.634490 14311 ts_manager.cc:194] Registered new tserver with Master: 9b23b4b59bcf4de59324bee5621357b3 (127.1.132.131:45331)
I20260923 22:14:54.634966 14311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:55977
I20260923 22:14:54.640936 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:14:54.653224 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.645647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.1.132.190:35105)
0923 22:14:54.645784 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.645786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.645903 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:14:54.645958 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.645959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.645971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.646055 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:54.646808 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.647606 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.648703 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.649032 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.649092 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.650442 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:54.650443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:54.650444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:54.650451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:54.652048 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:54.652431 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:54.652432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:54.652433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:54.652494 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:54.652675 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:54.652676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:54.652676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:54.652803 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:54.652896 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.653031 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.653110 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260923 22:14:54.655454 14311 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:14:54.657696 14311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:14:54.667688 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.662298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.1.132.130:44233)
0923 22:14:54.662415 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.662419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.662769 (+ 350us) server_negotiation.cc:407] Connection header received
0923 22:14:54.662857 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.662859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.662876 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.662948 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:54.663206 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.664630 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.666145 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.667208 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.667253 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.667310 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.667558 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.667600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260923 22:14:54.668223 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.661717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:44233 (local address 127.0.0.1:59028)
0923 22:14:54.662655 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.662707 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:54.662781 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:54.663039 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:54.663042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:54.663044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:54.663116 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.663118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.664714 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.664714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.666037 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.666040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.667368 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.667368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.667452 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.667464 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:14:54.668025 (+ 561us) client_negotiation.cc:240] Negotiation successful
0923 22:14:54.668099 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":103,"threads_started":1}
I20260923 22:14:54.669189 14578 tablet_service.cc:1467] Processing CreateTablet for tablet cecc5df4b2794fb6923b86815b4fa90e (DEFAULT_TABLE table=test-table [id=1d4cb787e9ca4675aee848c9d5da0921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:54.669574 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cecc5df4b2794fb6923b86815b4fa90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:54.674856 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.661722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45331 (local address 127.0.0.1:52182)
0923 22:14:54.662889 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.662897 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:54.663069 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:54.663527 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:54.663527 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:54.663530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:54.663573 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.663575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.664981 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.664981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.666075 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.666078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.674557 (+ 8479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.674559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.674638 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.674643 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:14:54.674708 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:14:54.674759 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1155,"mutex_wait_us":4,"thread_start_us":121,"threads_started":1}
I20260923 22:14:54.674894 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.662009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.1.132.131:45331)
0923 22:14:54.662139 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.662144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.662946 (+ 802us) server_negotiation.cc:407] Connection header received
0923 22:14:54.663353 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.663356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.663372 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.663418 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:54.663746 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.664825 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.673506 (+ 8681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.674361 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.674420 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.674484 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.674752 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.674807 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":50,"threads_started":1}
I20260923 22:14:54.676260 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.661985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45723 (local address 127.0.0.1:60028)
0923 22:14:54.662985 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.662992 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:54.663026 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:54.663625 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:54.663626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:54.663627 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:54.663657 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.663658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.667313 (+ 3655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.667314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.668418 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.668421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.675942 (+ 7521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.675943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.676038 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.676045 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:14:54.676102 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:14:54.676160 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":65,"threads_started":1}
I20260923 22:14:54.676564 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.662431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.1.132.129:45723)
0923 22:14:54.662569 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.662573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.663114 (+ 541us) server_negotiation.cc:407] Connection header received
0923 22:14:54.663149 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.663151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.663165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.663228 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:54.666175 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.667207 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.668516 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.675844 (+ 7328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.675927 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.675982 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.676435 (+ 453us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.676488 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260923 22:14:54.676723 14703 tablet_service.cc:1467] Processing CreateTablet for tablet cecc5df4b2794fb6923b86815b4fa90e (DEFAULT_TABLE table=test-table [id=1d4cb787e9ca4675aee848c9d5da0921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:54.677098 14703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cecc5df4b2794fb6923b86815b4fa90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:54.677177 14444 tablet_service.cc:1467] Processing CreateTablet for tablet cecc5df4b2794fb6923b86815b4fa90e (DEFAULT_TABLE table=test-table [id=1d4cb787e9ca4675aee848c9d5da0921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:14:54.677489 14444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cecc5df4b2794fb6923b86815b4fa90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:54.680344 14811 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: Bootstrap starting.
I20260923 22:14:54.681003 14811 tablet_bootstrap.cc:654] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:54.681337 14813 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: Bootstrap starting.
I20260923 22:14:54.681354 14811 log.cc:824] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:54.681937 14813 tablet_bootstrap.cc:654] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:54.682242 14813 log.cc:824] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:54.682566 14812 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: Bootstrap starting.
I20260923 22:14:54.683218 14812 tablet_bootstrap.cc:654] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:54.683674 14813 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: No bootstrap required, opened a new log
I20260923 22:14:54.684015 14812 log.cc:824] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:54.684154 14813 ts_tablet_manager.cc:1397] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:14:54.686895 14813 raft_consensus.cc:348] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.687098 14813 raft_consensus.cc:374] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:54.687134 14813 raft_consensus.cc:729] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd90daf3b54641a48d8e3fb34e26a265, State: Initialized, Role: FOLLOWER
I20260923 22:14:54.687552 14813 consensus_queue.cc:260] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.688127 14812 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: No bootstrap required, opened a new log
I20260923 22:14:54.688239 14812 ts_tablet_manager.cc:1397] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 22:14:54.690567 14811 tablet_bootstrap.cc:492] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: No bootstrap required, opened a new log
I20260923 22:14:54.690534 14812 raft_consensus.cc:348] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.690687 14811 ts_tablet_manager.cc:1397] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260923 22:14:54.691026 14813 ts_tablet_manager.cc:1428] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260923 22:14:54.691121 14812 raft_consensus.cc:374] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:54.691310 14510 heartbeater.cc:503] Master 127.1.132.190:35105 was elected leader, sending a full tablet report...
I20260923 22:14:54.692344 14812 raft_consensus.cc:729] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b23b4b59bcf4de59324bee5621357b3, State: Initialized, Role: FOLLOWER
I20260923 22:14:54.692574 14812 consensus_queue.cc:260] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.692920 14812 ts_tablet_manager.cc:1428] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260923 22:14:54.693066 14778 heartbeater.cc:503] Master 127.1.132.190:35105 was elected leader, sending a full tablet report...
I20260923 22:14:54.693328 14811 raft_consensus.cc:348] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.693542 14811 raft_consensus.cc:374] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:54.693610 14811 raft_consensus.cc:729] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35a6377503d41c6aab1f698008f0ada, State: Initialized, Role: FOLLOWER
I20260923 22:14:54.693775 14811 consensus_queue.cc:260] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.694072 14811 ts_tablet_manager.cc:1428] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:14:54.694218 14644 heartbeater.cc:503] Master 127.1.132.190:35105 was elected leader, sending a full tablet report...
I20260923 22:14:54.698877 14818 raft_consensus.cc:482] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:54.698997 14818 raft_consensus.cc:504] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.699376 14818 leader_election.cc:288] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a35a6377503d41c6aab1f698008f0ada (127.1.132.130:44233), bd90daf3b54641a48d8e3fb34e26a265 (127.1.132.129:45723)
I20260923 22:14:54.703667 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.699596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:53533 (local address 127.1.132.130:44233)
0923 22:14:54.699650 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.699652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.699777 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:14:54.699869 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.699870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.699879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.699905 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:54.700170 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.700957 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.702728 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.703300 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.703345 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.703353 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.703554 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.703595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:14:54.703679 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.699860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:35825 (local address 127.1.132.129:45723)
0923 22:14:54.699962 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.699963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.700023 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:14:54.700113 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.700114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.700122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.700149 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:14:54.700366 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.701080 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.702643 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.703200 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.703250 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.703260 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.703550 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.703598 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 22:14:54.703934 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.699710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45723 (local address 127.1.132.131:35825)
0923 22:14:54.699964 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.699973 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:54.700047 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:54.700231 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:54.700232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:54.700233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:54.700297 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.700299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.701200 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.701200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.702538 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.702541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.703338 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.703339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.703456 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.703462 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:14:54.703792 (+ 330us) client_negotiation.cc:240] Negotiation successful
0923 22:14:54.703837 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":52,"thread_start_us":58,"threads_started":1}
I20260923 22:14:54.704336 14464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cecc5df4b2794fb6923b86815b4fa90e" candidate_uuid: "9b23b4b59bcf4de59324bee5621357b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd90daf3b54641a48d8e3fb34e26a265" is_pre_election: true
I20260923 22:14:54.704511 14464 raft_consensus.cc:2454] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b23b4b59bcf4de59324bee5621357b3 in term 0.
I20260923 22:14:54.704697 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.699507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:44233 (local address 127.1.132.131:53533)
0923 22:14:54.699709 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.699720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:54.699801 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:54.700011 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:54.700012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:54.700014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:14:54.700088 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.700090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.701401 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.701402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.702631 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:54.702635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.703408 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.703409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:54.703480 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.703484 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:14:54.704075 (+ 591us) client_negotiation.cc:240] Negotiation successful
0923 22:14:54.704121 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260923 22:14:54.704751 14666 leader_election.cc:302] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b23b4b59bcf4de59324bee5621357b3, bd90daf3b54641a48d8e3fb34e26a265; no voters:
I20260923 22:14:54.704875 14818 raft_consensus.cc:2781] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:54.704929 14818 raft_consensus.cc:482] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:54.704949 14818 raft_consensus.cc:3037] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:54.705103 14598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cecc5df4b2794fb6923b86815b4fa90e" candidate_uuid: "9b23b4b59bcf4de59324bee5621357b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35a6377503d41c6aab1f698008f0ada" is_pre_election: true
I20260923 22:14:54.705248 14598 raft_consensus.cc:2454] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b23b4b59bcf4de59324bee5621357b3 in term 0.
I20260923 22:14:54.705767 14818 raft_consensus.cc:504] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.705883 14818 leader_election.cc:288] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [CANDIDATE]: Term 1 election: Requested vote from peers a35a6377503d41c6aab1f698008f0ada (127.1.132.130:44233), bd90daf3b54641a48d8e3fb34e26a265 (127.1.132.129:45723)
I20260923 22:14:54.706120 14598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cecc5df4b2794fb6923b86815b4fa90e" candidate_uuid: "9b23b4b59bcf4de59324bee5621357b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35a6377503d41c6aab1f698008f0ada"
I20260923 22:14:54.706149 14464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cecc5df4b2794fb6923b86815b4fa90e" candidate_uuid: "9b23b4b59bcf4de59324bee5621357b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd90daf3b54641a48d8e3fb34e26a265"
I20260923 22:14:54.706194 14598 raft_consensus.cc:3037] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:54.706218 14464 raft_consensus.cc:3037] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:54.706959 14464 raft_consensus.cc:2454] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b23b4b59bcf4de59324bee5621357b3 in term 1.
I20260923 22:14:54.707003 14598 raft_consensus.cc:2454] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b23b4b59bcf4de59324bee5621357b3 in term 1.
I20260923 22:14:54.707180 14666 leader_election.cc:302] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b23b4b59bcf4de59324bee5621357b3, bd90daf3b54641a48d8e3fb34e26a265; no voters:
I20260923 22:14:54.707285 14818 raft_consensus.cc:2781] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:54.707444 14818 raft_consensus.cc:686] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 1 LEADER]: Becoming Leader. State: Replica: 9b23b4b59bcf4de59324bee5621357b3, State: Running, Role: LEADER
I20260923 22:14:54.707531 14818 consensus_queue.cc:237] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } }
I20260923 22:14:54.708392 14311 catalog_manager.cc:5495] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b23b4b59bcf4de59324bee5621357b3 (127.1.132.131). New cstate: current_term: 1 leader_uuid: "9b23b4b59bcf4de59324bee5621357b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b23b4b59bcf4de59324bee5621357b3" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45331 } health_report { overall_health: HEALTHY } } }
I20260923 22:14:54.716089 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.712966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.1.132.131:45331)
0923 22:14:54.713004 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:54.713006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:54.713015 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:54.713084 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:54.713085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:54.713104 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:54.713177 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:14:54.713342 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.714061 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.715119 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:54.715429 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:54.715480 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:54.715654 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:14:54.715780 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:14:54.715847 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:54.715995 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 22:14:54.716015 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:14:54.718192 14464 raft_consensus.cc:1261] T cecc5df4b2794fb6923b86815b4fa90e P bd90daf3b54641a48d8e3fb34e26a265 [term 1 FOLLOWER]: Refusing update from remote peer 9b23b4b59bcf4de59324bee5621357b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:54.718287 14598 raft_consensus.cc:1261] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 1 FOLLOWER]: Refusing update from remote peer 9b23b4b59bcf4de59324bee5621357b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:54.718564 14822 consensus_queue.cc:1035] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd90daf3b54641a48d8e3fb34e26a265" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:54.718734 14822 consensus_queue.cc:1035] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35a6377503d41c6aab1f698008f0ada" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:54.721626 14825 mvcc.cc:205] Tried to move back new op lower bound from 7332666141563219968 to 7332666141522546688. Current Snapshot: MvccSnapshot[applied={T|T < 7332666141563219968}]
I20260923 22:14:54.721900 14828 mvcc.cc:205] Tried to move back new op lower bound from 7332666141563219968 to 7332666141522546688. Current Snapshot: MvccSnapshot[applied={T|T < 7332666141563219968}]
I20260923 22:14:54.727530 14311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:54.727622 14311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:14:54.728397 14826 mvcc.cc:205] Tried to move back new op lower bound from 7332666141563219968 to 7332666141522546688. Current Snapshot: MvccSnapshot[applied={T|T < 7332666141563219968}]
I20260923 22:14:54.728549 14311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5833959ae41f44d786c60a4f8833823b: Sending DeleteTablet for 3 replicas of tablet cecc5df4b2794fb6923b86815b4fa90e
I20260923 22:14:54.729092 14444 tablet_service.cc:1514] Processing DeleteTablet for tablet cecc5df4b2794fb6923b86815b4fa90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:54 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:60028
I20260923 22:14:54.729204 14578 tablet_service.cc:1514] Processing DeleteTablet for tablet cecc5df4b2794fb6923b86815b4fa90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:54 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:59028
I20260923 22:14:54.729432 14703 tablet_service.cc:1514] Processing DeleteTablet for tablet cecc5df4b2794fb6923b86815b4fa90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:14:54 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:52182
I20260923 22:14:54.729568 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14350
I20260923 22:14:54.736302 14835 tablet_replica.cc:330] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada: stopping tablet replica
I20260923 22:14:54.736301 14836 tablet_replica.cc:330] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3: stopping tablet replica
I20260923 22:14:54.736472 14836 raft_consensus.cc:2229] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:14:54.736508 14835 raft_consensus.cc:2229] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:14:54.736668 14835 raft_consensus.cc:2258] T cecc5df4b2794fb6923b86815b4fa90e P a35a6377503d41c6aab1f698008f0ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:14:54.736698 14836 raft_consensus.cc:2258] T cecc5df4b2794fb6923b86815b4fa90e P 9b23b4b59bcf4de59324bee5621357b3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:14:54.736838 14298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:45723 (error 108)
I20260923 22:14:54.737221 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14512
I20260923 22:14:54.743347 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14647
I20260923 22:14:54.744290 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.744132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45723 (local address 127.0.0.1:60044)
0923 22:14:54.744211 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.744251 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:45723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:14:54.744434 14298 catalog_manager.cc:4584] TS bd90daf3b54641a48d8e3fb34e26a265 (127.1.132.129:45723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cecc5df4b2794fb6923b86815b4fa90e: Network error: Client connection negotiation failed: client connection to 127.1.132.129:45723: connect: Connection refused (error 111)
I20260923 22:14:54.744732 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.744636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:44233 (local address 127.0.0.1:59030)
0923 22:14:54.744686 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.744707 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.130:44233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 22:14:54.749670 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14279
I20260923 22:14:54.749795 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:54.749663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45331 (local address 127.0.0.1:52190)
0923 22:14:54.749733 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:54.749757 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.131:45331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 22:14:54.754805 1554 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:14:55.089082 1554 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:14:55.089179 1554 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:14:55.424118 1554 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:14:55.424 UTC [13346] LOG: received smart shutdown request
2026-09-23 22:14:55.427 UTC [13346] LOG: background worker "logical replication launcher" (PID 13355) exited with exit code 1
2026-09-23 22:14:55.428 UTC [13350] LOG: shutting down
2026-09-23 22:14:55.641 UTC [13346] LOG: database system is shut down
2026-09-23T22:14:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48619 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:55 dist-test-slave-l4q5 krb5kdc[14846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:55 dist-test-slave-l4q5 krb5kdc[14846](info): set up 2 sockets
Sep 23 22:14:55 dist-test-slave-l4q5 krb5kdc[14846](info): commencing operation
krb5kdc: starting...
W20260923 22:14:56.715929 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:14:56 dist-test-slave-l4q5 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:14:56.838999 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:38121
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:56.920789 14859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:56.921018 14859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:56.921041 14859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:56.922243 14859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:56.922286 14859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:56.922298 14859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:56.922309 14859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:56.922319 14859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:56.923913 14859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:38121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:56.924211 14859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:56.924448 14859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:56.929375 14864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:56.929361 14865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:56.929339 14859 server_base.cc:1034] running on GCE node
W20260923 22:14:56.929397 14867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:56.929988 14859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:56.930320 14859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:56.931491 14859 hybrid_clock.cc:648] HybridClock initialized: now 1790201696931477 us; error 32 us; skew 500 ppm
Sep 23 22:14:56 dist-test-slave-l4q5 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:56.941632 14859 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:14:56.942094 14859 webserver.cc:460] Webserver started at http://127.1.132.190:40201/ using document root <none> and password file <none>
I20260923 22:14:56.942281 14859 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:56.942322 14859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:56.942416 14859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:56.943524 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d"
format_stamp: "Formatted at 2026-09-23 22:14:56 on dist-test-slave-l4q5"
server_key: "b94391382a4bf63c55cbbb68ba4860a0"
server_key_iv: "640ec53d3c9a083215bea1054658b05b"
server_key_version: "encryptionkey@0"
I20260923 22:14:56.943830 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d"
format_stamp: "Formatted at 2026-09-23 22:14:56 on dist-test-slave-l4q5"
server_key: "b94391382a4bf63c55cbbb68ba4860a0"
server_key_iv: "640ec53d3c9a083215bea1054658b05b"
server_key_version: "encryptionkey@0"
I20260923 22:14:56.945204 14859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:14:56.946080 14874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:56.946295 14859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:14:56.946381 14859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d"
format_stamp: "Formatted at 2026-09-23 22:14:56 on dist-test-slave-l4q5"
server_key: "b94391382a4bf63c55cbbb68ba4860a0"
server_key_iv: "640ec53d3c9a083215bea1054658b05b"
server_key_version: "encryptionkey@0"
I20260923 22:14:56.946445 14859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:56.967540 14859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:56.970170 14859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:56.970296 14859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:56.974439 14859 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:38121
I20260923 22:14:56.974542 14926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:38121 every 8 connection(s)
I20260923 22:14:56.974913 14859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:56.976639 14927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:56.978394 14927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:56.984333 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14859
I20260923 22:14:56.984448 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
Sep 23 22:14:56 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:56.984629 1554 external_mini_cluster.cc:1428] Setting key 9369bb120061dc167fe1914290624a8a
I20260923 22:14:56.988971 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.979455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47062 (local address 127.1.132.190:38121)
0923 22:14:56.979694 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:56.979699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:56.979740 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:14:56.980122 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:56.980123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:56.980141 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:56.980250 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:56.980810 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:56.981613 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:56.982464 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:56.982775 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:56.982870 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:56.985072 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:56.985081 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:56.985081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:56.985090 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:56.987188 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:56.987656 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:56.987659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:56.987660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:56.987757 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:56.987989 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:56.987989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:56.987990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:56.988288 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:56.988368 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:56.988620 (+ 252us) server_negotiation.cc:299] Negotiation successful
0923 22:14:56.988739 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":177,"threads_started":1}
I20260923 22:14:56.988978 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.979187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38121 (local address 127.0.0.1:47062)
0923 22:14:56.979371 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.979422 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:56.979971 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:56.980253 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:56.980255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:56.980546 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:56.980713 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:56.980717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:56.981709 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:56.981711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:56.982381 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:56.982383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:56.982850 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:56.982851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:56.982915 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:56.983441 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:56.983449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:56.984984 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:56.987274 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:56.987282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:56.987284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:56.987552 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:56.987827 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:56.987827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:56.987828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:56.987929 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:56.988355 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:56.988362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:56.988444 (+ 82us) client_negotiation.cc:769] Sending connection context
0923 22:14:56.988563 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 22:14:56.988831 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":91,"thread_start_us":79,"threads_started":1}
I20260923 22:14:56.989980 14927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:56.990365 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.990243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47196)
0923 22:14:56.990289 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.990342 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:14:56.990512 14875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
I20260923 22:14:56.990990 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:56.990863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47204)
0923 22:14:56.990931 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:56.990968 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260923 22:14:56.991101 14927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:39831: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
I20260923 22:14:57.011739 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:39831
--webserver_interface=127.1.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:57.040297 14927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } attempt: 1
I20260923 22:14:57.040791 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.040660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47208)
0923 22:14:57.040726 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.040759 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:14:57.041091 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.041012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47212)
0923 22:14:57.041047 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.041071 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:14:57.041174 14927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:39831: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
W20260923 22:14:57.094704 14934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:57.094965 14934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:57.095000 14934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:57.096290 14934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:57.096335 14934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:57.096354 14934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:57.096369 14934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:57.096385 14934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:57.098001 14934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:57.098258 14934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:57.098487 14934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:57.103207 14942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:57.103188 14940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:57.103241 14934 server_base.cc:1034] running on GCE node
W20260923 22:14:57.103169 14939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:57.103600 14934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:57.103871 14934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:57.105034 14934 hybrid_clock.cc:648] HybridClock initialized: now 1790201697105033 us; error 39 us; skew 500 ppm
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:57.107537 14934 init.cc:377] Logged in from keytab as kudu/127.1.132.189@KRBTEST.COM (short username kudu)
I20260923 22:14:57.107960 14934 webserver.cc:460] Webserver started at http://127.1.132.189:35849/ using document root <none> and password file <none>
I20260923 22:14:57.108173 14934 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:57.108256 14934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:57.108384 14934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:57.109314 14927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } attempt: 2
I20260923 22:14:57.109551 14934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data/instance:
uuid: "c7aaf7e650074929a0aef80d70758336"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "7498423420c570d93bd983fae84f8e6d"
server_key_iv: "a875c3aeb989482088bfe3f95f47f26a"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.109697 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.109566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47226)
0923 22:14:57.109625 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.109660 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:14:57.109931 14934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal/instance:
uuid: "c7aaf7e650074929a0aef80d70758336"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "7498423420c570d93bd983fae84f8e6d"
server_key_iv: "a875c3aeb989482088bfe3f95f47f26a"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.110050 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.109958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47232)
0923 22:14:57.110000 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.110028 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:14:57.110123 14927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:39831: Network error: Client connection negotiation failed: client connection to 127.1.132.189:39831: connect: Connection refused (error 111)
I20260923 22:14:57.111279 14934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:14:57.111970 14949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:57.112169 14934 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:14:57.112289 14934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "c7aaf7e650074929a0aef80d70758336"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "7498423420c570d93bd983fae84f8e6d"
server_key_iv: "a875c3aeb989482088bfe3f95f47f26a"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.112360 14934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:57.126624 14934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:57.128567 14934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:57.128682 14934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:57.132280 14934 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.189:39831
I20260923 22:14:57.132320 15001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.189:39831 every 8 connection(s)
I20260923 22:14:57.132658 14934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
I20260923 22:14:57.133502 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:57.135018 15002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:57.136682 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 14934
I20260923 22:14:57.136781 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-1/wal/instance
I20260923 22:14:57.136968 1554 external_mini_cluster.cc:1428] Setting key 5eb2681e0aef5af311f3a9d0c265a447
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.188@KRBTEST.COM; defaulting to no policy
I20260923 22:14:57.144057 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.135788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.1.132.190:38121)
0923 22:14:57.135883 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.135885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.136061 (+ 176us) server_negotiation.cc:407] Connection header received
0923 22:14:57.136601 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.136602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.136613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.136664 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.137454 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.137937 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.138984 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.139192 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.139246 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.141526 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.141527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.141528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.141537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.142791 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.143167 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.143169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.143169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.143220 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.143432 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.143433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.143433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.143548 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.143602 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.143880 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.143964 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 22:14:57.144266 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.135709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38121 (local address 127.0.0.1:47068)
0923 22:14:57.135922 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.135958 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.136495 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.136738 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.136742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.137092 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.137370 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.137376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.138033 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.138034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.138888 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.138891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.139268 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.139268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.139409 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.139885 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.139903 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.141427 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.142874 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.142881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.142883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.143103 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.143274 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.143274 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.143275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.143378 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.143605 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.143612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.143763 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.143851 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.144021 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260923 22:14:57.144835 15002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.188@KRBTEST.COM" created.
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:14:57.152031 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.145095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47242)
0923 22:14:57.145144 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.145154 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.145192 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.145422 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.145423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.145571 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.145622 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.145623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.146642 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.146643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.147161 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.147163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.147759 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.147760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.147836 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.148251 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.148255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.149506 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.151110 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.151111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.151111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.151260 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.151425 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.151425 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.151426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.151476 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.151720 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.151721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.151759 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.151817 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.151913 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:14:57.152107 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.145132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.1.132.189:39831)
0923 22:14:57.145272 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.145277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.145295 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:14:57.145316 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.145318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.145331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.145393 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.145710 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.146556 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.147233 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.147670 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.147742 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.149577 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.149586 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.149586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.149594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.151035 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.151322 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.151325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.151326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.151375 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.151524 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.151525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.151525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.151665 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.151733 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.151845 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.151983 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260923 22:14:57.152642 15002 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:57.152953 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.152846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46388)
0923 22:14:57.152887 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.152930 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:14:57.153075 14951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
I20260923 22:14:57.153498 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.153408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46404)
0923 22:14:57.153449 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.153474 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:14:57.153673 15002 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:35487: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
I20260923 22:14:57.164846 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:35487
--webserver_interface=127.1.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:14:57.201268 14927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } attempt: 3
I20260923 22:14:57.202905 15002 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } attempt: 1
I20260923 22:14:57.203267 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.203159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46416)
0923 22:14:57.203207 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.203239 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:14:57.203655 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.203535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46426)
0923 22:14:57.203616 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.203637 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260923 22:14:57.203722 15002 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:35487: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:14:57.209468 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.201641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47256)
0923 22:14:57.201712 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.201728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.201796 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.201939 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.201939 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.202126 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.202202 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.202206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.202906 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.202907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.203725 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.203728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.204167 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.204168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.204297 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.204763 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.204768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.206289 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.208126 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.208127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.208127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.208375 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.208667 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.208668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.208668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.208740 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.209025 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.209027 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.209119 (+ 92us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.209211 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.209348 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:14:57.209534 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.201703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.1.132.189:39831)
0923 22:14:57.201758 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.201759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.201774 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:14:57.201858 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.201858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.201869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.201917 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.202284 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.202824 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.203816 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.204071 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.204141 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.206372 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.206373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.206374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.206396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.208047 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.208476 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.208478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.208479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.208555 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.208822 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.208823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.208823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.208965 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.209029 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.209278 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.209395 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:14:57.210144 14927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:14:57.210533 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.210407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46434)
0923 22:14:57.210471 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.210507 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:14:57.210911 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.210748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46448)
0923 22:14:57.210863 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.210885 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
W20260923 22:14:57.211159 14927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:35487: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
I20260923 22:14:57.242357 14927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } attempt: 1
I20260923 22:14:57.242866 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.242714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46450)
0923 22:14:57.242775 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.242823 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:14:57.243602 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.243456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46454)
0923 22:14:57.243546 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.243577 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260923 22:14:57.243708 14927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:35487: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
W20260923 22:14:57.252146 15009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:57.252401 15009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:57.252424 15009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:57.253595 15009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:57.253635 15009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:57.253648 15009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:57.253659 15009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:57.253669 15009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:57.255204 15009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:38121,127.1.132.189:39831,127.1.132.188:35487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:35487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:57.255424 15009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:57.255621 15009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:57.260794 15016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:57.260794 15015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:57.260800 15018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:57.261056 15009 server_base.cc:1034] running on GCE node
I20260923 22:14:57.261303 15009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:57.261601 15009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:57.262770 15009 hybrid_clock.cc:648] HybridClock initialized: now 1790201697262746 us; error 32 us; skew 500 ppm
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:57.265425 15009 init.cc:377] Logged in from keytab as kudu/127.1.132.188@KRBTEST.COM (short username kudu)
I20260923 22:14:57.265942 15009 webserver.cc:460] Webserver started at http://127.1.132.188:34011/ using document root <none> and password file <none>
I20260923 22:14:57.266142 15009 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:57.266185 15009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:57.266307 15009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:57.267379 15009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data/instance:
uuid: "b0856d18071d42dbbac930f5cdb15e8f"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "da344cc40e7c551023fc052f5812792d"
server_key_iv: "c2b8fe8707908899380375a5fabccfa8"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.267709 15009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal/instance:
uuid: "b0856d18071d42dbbac930f5cdb15e8f"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "da344cc40e7c551023fc052f5812792d"
server_key_iv: "c2b8fe8707908899380375a5fabccfa8"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.269018 15009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:14:57.269789 15025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:57.269970 15009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:14:57.270051 15009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "b0856d18071d42dbbac930f5cdb15e8f"
format_stamp: "Formatted at 2026-09-23 22:14:57 on dist-test-slave-l4q5"
server_key: "da344cc40e7c551023fc052f5812792d"
server_key_iv: "c2b8fe8707908899380375a5fabccfa8"
server_key_version: "encryptionkey@0"
I20260923 22:14:57.270123 15009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:57.271864 15002 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } attempt: 2
I20260923 22:14:57.272287 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.272157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46462)
0923 22:14:57.272222 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.272257 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:14:57.272667 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.272543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46478)
0923 22:14:57.272598 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.272636 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:14:57.272804 15002 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:35487: Network error: Client connection negotiation failed: client connection to 127.1.132.188:35487: connect: Connection refused (error 111)
I20260923 22:14:57.283854 15009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:57.285984 15009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:57.286109 15009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:57.290184 15009 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.188:35487
I20260923 22:14:57.290217 15077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.188:35487 every 8 connection(s)
I20260923 22:14:57.290668 15009 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
I20260923 22:14:57.291517 15078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:57.293169 15078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:57.299970 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15009
I20260923 22:14:57.300058 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201541420007-1554-0/minicluster-data/master-2/wal/instance
I20260923 22:14:57.300307 1554 external_mini_cluster.cc:1428] Setting key f01e66ee24567f3a09d62f0572385307
I20260923 22:14:57.302233 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.293912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.1.132.190:38121)
0923 22:14:57.293977 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.293979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.294149 (+ 170us) server_negotiation.cc:407] Connection header received
0923 22:14:57.294601 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.294602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.294612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.294697 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.295298 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.295785 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.296810 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.297021 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.297095 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.299240 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.299241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.299242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.299248 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.300809 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.301276 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.301278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.301278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.301345 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.301562 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.301562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.301563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.301731 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.301780 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.302107 (+ 327us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.302168 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:14:57.302469 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.293862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:38121 (local address 127.0.0.1:47078)
0923 22:14:57.294028 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.294070 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.294516 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.294751 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.294755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.295017 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.295224 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.295229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.295874 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.295875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.296724 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.296727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.297119 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.297121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.297238 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.297750 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.297758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.299141 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.300902 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.300906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.300909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.301206 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.301412 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.301412 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.301412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.301512 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.301788 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.301793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.301944 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.302084 (+ 140us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.302244 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260923 22:14:57.303010 15078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:14:57.309314 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.303334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.1.132.189:39831)
0923 22:14:57.303385 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.303387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.303396 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:57.303420 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.303421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.303431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.303484 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.303821 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.304459 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.305085 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.305363 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.305432 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.307000 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.307002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.307002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.307011 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.308303 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308607 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.308608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.308609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.308665 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308833 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.308834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.308834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.308950 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.309012 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.309149 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.309225 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:14:57.309310 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.303271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:39831 (local address 127.0.0.1:47266)
0923 22:14:57.303316 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.303327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.303367 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.303493 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.303493 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.303668 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.303724 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.303726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.304524 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.304525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.305006 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.305008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.305429 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.305430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.305501 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.305856 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.305861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.306918 (+ 1057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.308364 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.308366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.308545 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.308719 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.308720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.308777 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.309003 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.309004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.309045 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.309117 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.309211 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:14:57.309403 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.300700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46480 (local address 127.1.132.188:35487)
0923 22:14:57.300846 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.300851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.300873 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:14:57.300918 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.300920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.300935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.301007 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.301616 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.302607 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.303485 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.303920 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.303967 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.306410 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.306421 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.306421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.306428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.308100 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308509 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.308512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.308512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.308569 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.308756 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.308757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.308757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.308933 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.309094 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.309140 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.309286 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260923 22:14:57.309813 15078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:57.316640 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.310076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46484)
0923 22:14:57.310115 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.310124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.310164 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.310262 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.310263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.310399 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.310430 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.310431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.311173 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.311174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.311777 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.311779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.312259 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.312260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.312305 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.312712 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.312717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.314015 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.315534 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.315535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.315536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.315722 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.315963 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.315963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.315963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.316016 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.316302 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.316303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.316342 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.316401 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.316507 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:14:57.316640 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.310113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.1.132.188:35487)
0923 22:14:57.310147 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.310149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.310156 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:14:57.310207 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.310208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.310217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.310262 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.310491 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.311093 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.311843 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.312207 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.312261 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.314116 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.314117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.314118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.314124 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.315460 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.315789 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.315790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.315791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.315846 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.316074 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.316075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.316076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.316238 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.316309 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.316447 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.316548 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":17}
I20260923 22:14:57.317420 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.310182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.1.132.190:38121)
0923 22:14:57.310221 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.310223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.310239 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:14:57.310263 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.310264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.310273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.310319 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.310835 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.311414 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.312174 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.312432 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.312514 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.314690 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.314691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.314691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.314698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.316345 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.316696 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.316697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.316698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.316754 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.316944 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.316945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.316945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.317058 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.317226 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.317260 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.317335 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:14:57.318467 15078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: Bootstrap starting.
I20260923 22:14:57.318852 14927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } attempt: 2
I20260923 22:14:57.319471 15078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:57.319752 15078 log.cc:824] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:57.320516 15078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: No bootstrap required, opened a new log
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:14:57.322558 15078 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.322748 15078 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:57.322813 15078 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0856d18071d42dbbac930f5cdb15e8f, State: Initialized, Role: FOLLOWER
I20260923 22:14:57.322957 15078 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201696, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:14:57.323225 15084 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.323288 15078 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:57.323297 15084 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:14:57.325225 15095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:57.325346 15095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:57.326073 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.318091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.1.132.189:39831)
0923 22:14:57.318136 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.318138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.318146 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:57.318171 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.318172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.318182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.318230 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.318781 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.319512 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.320479 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.320683 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.320738 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.323104 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.323106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.323106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.323115 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.324710 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325090 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.325092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.325092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.325153 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325347 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.325348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.325348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.325497 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.325675 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.325894 (+ 219us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.325981 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:57.326392 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.319095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46496)
0923 22:14:57.319139 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.319148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.319180 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.319292 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.319293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.319433 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.319476 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.319478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.320448 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.320450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.321150 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.321154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.321595 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.321597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.321659 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.322059 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.322064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.323429 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.325322 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.325323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.325523 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.325733 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.325734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.325793 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.326114 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.326115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.326164 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.326212 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.326304 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:14:57.326757 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.319150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.1.132.188:35487)
0923 22:14:57.319188 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.319189 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.319197 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:14:57.319218 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.319218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.319226 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.319283 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.319558 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.320353 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.321227 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.321513 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.321591 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.323528 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.323530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.323530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.323537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.325234 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325605 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.325606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.325606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.325662 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.325860 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.325861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.325861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.326032 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.326556 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.326598 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.326681 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:14:57.328003 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:14:57.325415 15009 master_runner.cc:186] Error getting master registration for 127.1.132.190:38121: OK, instance_id { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" instance_seqno: 1790201696973029 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:14:57.328266 15009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:57.328455 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: Bootstrap starting.
I20260923 22:14:57.329192 14927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:57.329509 14927 log.cc:824] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:57.330389 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: No bootstrap required, opened a new log
I20260923 22:14:57.332410 14927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.332547 14927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:57.332576 14927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ab66ce1d81454c8cfaf6d8e0aecf2d, State: Initialized, Role: FOLLOWER
I20260923 22:14:57.332686 14927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.332995 14927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: configured and running, proceeding with master startup.
W20260923 22:14:57.335628 15115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:57.335788 15115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:57.335948 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.336089 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:57.337850 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.329675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.1.132.189:39831)
0923 22:14:57.329748 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.329750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.330023 (+ 273us) server_negotiation.cc:407] Connection header received
0923 22:14:57.330117 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.330118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.330128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.330155 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.330981 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.331754 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.332795 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.333058 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.333103 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.334701 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.334703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.334703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.334712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.336345 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.336742 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.336743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.336744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.336815 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.337033 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.337034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.337034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.337173 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.337273 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.337678 (+ 405us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.337758 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 22:14:57.338009 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.329636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.1.132.190:38121)
0923 22:14:57.329715 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.329717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.330004 (+ 287us) server_negotiation.cc:407] Connection header received
0923 22:14:57.330105 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.330106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.330117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.330147 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.330980 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.331708 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.332803 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.333089 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.333133 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.334870 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.334871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.334871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.334878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.336820 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.337199 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.337200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.337201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.337258 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.337451 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.337451 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.337452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.337577 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.337653 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.337848 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.337923 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 22:14:57.335812 14859 master_runner.cc:186] Error getting master registration for 127.1.132.189:39831: OK, instance_id { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" instance_seqno: 1790201697131249 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:14:57.338304 14859 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:57.345817 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.334773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.1.132.188:35487)
0923 22:14:57.334813 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.334815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.335078 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:14:57.335556 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.335557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.335570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.335599 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.338568 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.339280 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.340448 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.340765 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.340811 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.342480 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.342481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.342482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.342491 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.344362 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.344821 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.344823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.344823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.344888 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.345105 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.345106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.345106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.345250 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.345334 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.345580 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.345671 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:14:57.363960 15002 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } attempt: 3
Sep 23 22:14:57 dist-test-slave-l4q5 krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201697, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:14:57.372702 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.364316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:35487 (local address 127.0.0.1:46520)
0923 22:14:57.364368 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:14:57.364379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:14:57.364417 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:14:57.364565 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:14:57.364566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:14:57.364757 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:14:57.364816 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.364818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.365740 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.365741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.366605 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:14:57.366608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.367130 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.367131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:14:57.367202 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.367743 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:14:57.367748 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:14:57.369346 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:14:57.371266 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.371267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.371268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.371511 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.371736 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:14:57.371737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:14:57.371737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:14:57.371820 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:14:57.372121 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:14:57.372122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:14:57.372171 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:14:57.372270 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:14:57.372378 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:14:57.373083 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:57.364367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.1.132.188:35487)
0923 22:14:57.364408 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:57.364410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:57.364419 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:14:57.364475 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:57.364475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:57.364486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:57.364546 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:57.364889 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.365650 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.366700 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:57.367011 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:57.367085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:57.369470 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:57.369471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:57.369472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:57.369481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:57.371150 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.371586 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.371587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.371588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.371652 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:57.371886 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:57.371887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:57.371887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:57.372037 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:57.372095 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:57.372902 (+ 807us) server_negotiation.cc:299] Negotiation successful
0923 22:14:57.372996 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:14:57.374670 15002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: Bootstrap starting.
I20260923 22:14:57.375422 15002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:57.375736 15002 log.cc:824] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:57.376682 15002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: No bootstrap required, opened a new log
I20260923 22:14:57.378779 15002 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.378960 15002 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:57.378995 15002 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7aaf7e650074929a0aef80d70758336, State: Initialized, Role: FOLLOWER
I20260923 22:14:57.379987 15002 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.380365 15119 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.380434 15002 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:57.380530 15119 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:14:57.382417 15130 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:57.382537 15130 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:57.384637 14934 master_runner.cc:263] No leader master found from master c7aaf7e650074929a0aef80d70758336
I20260923 22:14:57.384689 14934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:14:57.431473 15104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:14:57.431604 15104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.431841 15104 leader_election.cc:288] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7aaf7e650074929a0aef80d70758336 (127.1.132.189:39831), b0856d18071d42dbbac930f5cdb15e8f (127.1.132.188:35487)
I20260923 22:14:57.432389 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0856d18071d42dbbac930f5cdb15e8f" is_pre_election: true
I20260923 22:14:57.432564 15052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55ab66ce1d81454c8cfaf6d8e0aecf2d in term 0.
I20260923 22:14:57.433099 14876 leader_election.cc:302] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55ab66ce1d81454c8cfaf6d8e0aecf2d, b0856d18071d42dbbac930f5cdb15e8f; no voters:
I20260923 22:14:57.433347 15104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:14:57.433418 15104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:14:57.433441 15104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:57.435141 15104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.435320 15104 leader_election.cc:288] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [CANDIDATE]: Term 1 election: Requested vote from peers c7aaf7e650074929a0aef80d70758336 (127.1.132.189:39831), b0856d18071d42dbbac930f5cdb15e8f (127.1.132.188:35487)
I20260923 22:14:57.435819 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0856d18071d42dbbac930f5cdb15e8f"
I20260923 22:14:57.435914 15052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:57.436925 15052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55ab66ce1d81454c8cfaf6d8e0aecf2d in term 1.
I20260923 22:14:57.437184 14876 leader_election.cc:302] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55ab66ce1d81454c8cfaf6d8e0aecf2d, b0856d18071d42dbbac930f5cdb15e8f; no voters:
I20260923 22:14:57.437580 14976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7aaf7e650074929a0aef80d70758336" is_pre_election: true
I20260923 22:14:57.437759 14976 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55ab66ce1d81454c8cfaf6d8e0aecf2d in term 0.
I20260923 22:14:57.437891 14975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7aaf7e650074929a0aef80d70758336"
I20260923 22:14:57.437951 14975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:57.438870 14975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55ab66ce1d81454c8cfaf6d8e0aecf2d in term 1.
I20260923 22:14:57.439260 15104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:57.439430 15104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [term 1 LEADER]: Becoming Leader. State: Replica: 55ab66ce1d81454c8cfaf6d8e0aecf2d, State: Running, Role: LEADER
I20260923 22:14:57.439531 15104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } }
I20260923 22:14:57.439924 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ab66ce1d81454c8cfaf6d8e0aecf2d. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.439991 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:57.440335 15134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:57.440549 15134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:57.442787 15052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [term 1 FOLLOWER]: Refusing update from remote peer 55ab66ce1d81454c8cfaf6d8e0aecf2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:57.443073 14975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [term 1 FOLLOWER]: Refusing update from remote peer 55ab66ce1d81454c8cfaf6d8e0aecf2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:14:57.444398 15104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:57.446466 15119 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ab66ce1d81454c8cfaf6d8e0aecf2d. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.446564 15119 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:57.446722 15104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:14:57.448065 15084 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ab66ce1d81454c8cfaf6d8e0aecf2d. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.448173 15084 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:57.450294 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.450384 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:57.450857 15119 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.450948 15119 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:57.451220 15135 mvcc.cc:205] Tried to move back new op lower bound from 7332666152722337792 to 7332666152712921088. Current Snapshot: MvccSnapshot[applied={T|T < 7332666152722337792}]
I20260923 22:14:57.451781 15136 mvcc.cc:205] Tried to move back new op lower bound from 7332666152722337792 to 7332666152712921088. Current Snapshot: MvccSnapshot[applied={T|T < 7332666152722337792}]
I20260923 22:14:57.452171 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.452270 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:57.452800 15084 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55ab66ce1d81454c8cfaf6d8e0aecf2d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 38121 } } peers { permanent_uuid: "c7aaf7e650074929a0aef80d70758336" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 39831 } } peers { permanent_uuid: "b0856d18071d42dbbac930f5cdb15e8f" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 35487 } } }
I20260923 22:14:57.452888 15084 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:14:57.454128 15134 catalog_manager.cc:1347] Generated new cluster ID: c09f001f80ff4be0bb0458ba24d79dfc
I20260923 22:14:57.454175 15134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:14:57.456465 15138 mvcc.cc:205] Tried to move back new op lower bound from 7332666152722337792 to 7332666152712921088. Current Snapshot: MvccSnapshot[applied={T|T < 7332666152722337792 or (T in {7332666152722337792})}]
I20260923 22:14:57.471877 15134 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:57.472841 15134 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:57.478821 15134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55ab66ce1d81454c8cfaf6d8e0aecf2d: Generated new TSK 0
I20260923 22:14:57.479090 15134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:14:58.325839 15095 catalog_manager.cc:1259] Loaded cluster ID: c09f001f80ff4be0bb0458ba24d79dfc
I20260923 22:14:58.325914 15095 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: loading cluster ID for follower catalog manager: success
I20260923 22:14:58.326758 15095 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: acquiring CA information for follower catalog manager: success
I20260923 22:14:58.327076 15095 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b0856d18071d42dbbac930f5cdb15e8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:14:58.383014 15130 catalog_manager.cc:1259] Loaded cluster ID: c09f001f80ff4be0bb0458ba24d79dfc
I20260923 22:14:58.383095 15130 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: loading cluster ID for follower catalog manager: success
I20260923 22:14:58.383876 15130 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: acquiring CA information for follower catalog manager: success
I20260923 22:14:58.384127 15130 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7aaf7e650074929a0aef80d70758336: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:14:58.539873 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14859
I20260923 22:14:58.544348 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 14934
I20260923 22:14:58.548477 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15009
2026-09-23T22:14:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2812 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:58 dist-test-slave-l4q5 krb5kdc[15149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:58 dist-test-slave-l4q5 krb5kdc[15149](info): set up 2 sockets
Sep 23 22:14:58 dist-test-slave-l4q5 krb5kdc[15149](info): commencing operation
krb5kdc: starting...
W20260923 22:14:59.500141 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:14:59.624524 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:37435
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:37435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:14:59.705531 15162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:14:59.705763 15162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:14:59.705786 15162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:14:59.707000 15162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:14:59.707046 15162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:14:59.707059 15162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:14:59.707072 15162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:14:59.707082 15162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:14:59.708607 15162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:37435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:37435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:14:59.708843 15162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:14:59.709035 15162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:14:59.713799 15170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:59.713792 15168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:14:59.713788 15167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:14:59.713968 15162 server_base.cc:1034] running on GCE node
I20260923 22:14:59.714208 15162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:14:59.714486 15162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:14:59.715631 15162 hybrid_clock.cc:648] HybridClock initialized: now 1790201699715618 us; error 27 us; skew 500 ppm
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201699, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:14:59.718058 15162 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:14:59.718446 15162 webserver.cc:460] Webserver started at http://127.1.132.190:34499/ using document root <none> and password file <none>
I20260923 22:14:59.718705 15162 fs_manager.cc:359] Metadata directory not provided
I20260923 22:14:59.718755 15162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:14:59.718865 15162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:14:59.719873 15162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a"
format_stamp: "Formatted at 2026-09-23 22:14:59 on dist-test-slave-l4q5"
server_key: "736ee4247b3aa50dce82eab1aefd43ca"
server_key_iv: "3f5580ea6f88e7d1a1a45fd09a74dcf7"
server_key_version: "encryptionkey@0"
I20260923 22:14:59.720211 15162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a"
format_stamp: "Formatted at 2026-09-23 22:14:59 on dist-test-slave-l4q5"
server_key: "736ee4247b3aa50dce82eab1aefd43ca"
server_key_iv: "3f5580ea6f88e7d1a1a45fd09a74dcf7"
server_key_version: "encryptionkey@0"
I20260923 22:14:59.721562 15162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:14:59.722291 15177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:14:59.722458 15162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:14:59.722563 15162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a"
format_stamp: "Formatted at 2026-09-23 22:14:59 on dist-test-slave-l4q5"
server_key: "736ee4247b3aa50dce82eab1aefd43ca"
server_key_iv: "3f5580ea6f88e7d1a1a45fd09a74dcf7"
server_key_version: "encryptionkey@0"
I20260923 22:14:59.722630 15162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:14:59.736599 15162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:14:59.738912 15162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:14:59.739037 15162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:14:59.742836 15162 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:37435
I20260923 22:14:59.742884 15229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:37435 every 8 connection(s)
I20260923 22:14:59.743211 15162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:14:59.744091 15230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:14:59.746677 15230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: Bootstrap starting.
I20260923 22:14:59.747246 15230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: Neither blocks nor log segments found. Creating new log.
I20260923 22:14:59.747494 15230 log.cc:824] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:14:59.748241 15230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: No bootstrap required, opened a new log
I20260923 22:14:59.749382 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15162
I20260923 22:14:59.749482 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:14:59.749658 1554 external_mini_cluster.cc:1428] Setting key 5944ce0e51108f27e4a8c09b84d769e0
I20260923 22:14:59.750176 15230 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } }
I20260923 22:14:59.750375 15230 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:14:59.750414 15230 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a4509c4f1d4bcebc4e0a9cbe2d908a, State: Initialized, Role: FOLLOWER
I20260923 22:14:59.750550 15230 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } }
I20260923 22:14:59.750628 15230 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:14:59.750666 15230 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:14:59.750749 15230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:14:59.751802 15230 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } }
I20260923 22:14:59.751889 15230 leader_election.cc:302] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a4509c4f1d4bcebc4e0a9cbe2d908a; no voters:
I20260923 22:14:59.752039 15230 leader_election.cc:288] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:14:59.752082 15235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:14:59.752234 15235 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [term 1 LEADER]: Becoming Leader. State: Replica: f3a4509c4f1d4bcebc4e0a9cbe2d908a, State: Running, Role: LEADER
I20260923 22:14:59.752327 15230 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:14:59.752383 15235 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } }
I20260923 22:14:59.752784 15235 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a4509c4f1d4bcebc4e0a9cbe2d908a. Latest consensus state: current_term: 1 leader_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } } }
I20260923 22:14:59.753033 15235 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [sys.catalog]: This master's current role is: LEADER
I20260923 22:14:59.752923 15236 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a4509c4f1d4bcebc4e0a9cbe2d908a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37435 } } }
I20260923 22:14:59.753428 15236 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a [sys.catalog]: This master's current role is: LEADER
W20260923 22:14:59.754025 15251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:14:59.754079 15251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:14:59.754119 15244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:14:59.754254 15244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:14:59.755718 15244 catalog_manager.cc:1347] Generated new cluster ID: 4ee3fe6b37c34fb4a0ccd18ea0c51cdf
I20260923 22:14:59.755764 15244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:14:59.761178 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:14:59.750205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.1.132.190:37435)
0923 22:14:59.750362 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:14:59.750370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:14:59.750395 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:14:59.750551 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:14:59.750557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:14:59.750602 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:14:59.750733 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:14:59.751520 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:59.752759 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:59.753920 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:14:59.754532 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:14:59.754643 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:14:59.757320 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:14:59.757324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:14:59.757325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:14:59.757337 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:14:59.759279 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:59.759667 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:59.759670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:59.759671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:59.759819 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:14:59.760139 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:14:59.760140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:14:59.760141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:14:59.760502 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:14:59.760657 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 22:14:59.760802 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:14:59.760962 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260923 22:14:59.762153 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:14:59.775158 15244 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:14:59.775710 15244 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:14:59.783730 15244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3a4509c4f1d4bcebc4e0a9cbe2d908a: Generated new TSK 0
I20260923 22:14:59.783928 15244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:14:59.816898 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15162
2026-09-23T22:14:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1267 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15257](info): set up 2 sockets
Sep 23 22:14:59 dist-test-slave-l4q5 krb5kdc[15257](info): commencing operation
krb5kdc: starting...
W20260923 22:15:00.768345 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:15:00 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:15:00.887943 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:00.971415 15270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:00.971635 15270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:00.971658 15270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:00.972892 15270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:00.972935 15270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:00.972949 15270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:00.972959 15270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:00.972970 15270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:00.974449 15270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:00.974678 15270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:00.974867 15270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:00.979875 15275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:00.980095 15270 server_base.cc:1034] running on GCE node
W20260923 22:15:00.979874 15276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:00.979921 15278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:00.980454 15270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:00.980928 15270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:00.982125 15270 hybrid_clock.cc:648] HybridClock initialized: now 1790201700982097 us; error 50 us; skew 500 ppm
Sep 23 22:15:00 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201700, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:00.984907 15270 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:15:00.985409 15270 webserver.cc:460] Webserver started at http://127.1.132.190:45175/ using document root <none> and password file <none>
I20260923 22:15:00.985615 15270 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:00.985657 15270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:00.985749 15270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:00.987668 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "16e431d93dae4c85acb5e4569b5bcdef"
format_stamp: "Formatted at 2026-09-23 22:15:00 on dist-test-slave-l4q5"
server_key: "1b8a2862ab34ad5b1204c70ed4b064c1"
server_key_iv: "629c2ff4a67671d26201fdd07c49e1fb"
server_key_version: "encryptionkey@0"
I20260923 22:15:00.987987 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "16e431d93dae4c85acb5e4569b5bcdef"
format_stamp: "Formatted at 2026-09-23 22:15:00 on dist-test-slave-l4q5"
server_key: "1b8a2862ab34ad5b1204c70ed4b064c1"
server_key_iv: "629c2ff4a67671d26201fdd07c49e1fb"
server_key_version: "encryptionkey@0"
I20260923 22:15:00.989545 15270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 22:15:00.990411 15285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:00.990697 15270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:00.990784 15270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "16e431d93dae4c85acb5e4569b5bcdef"
format_stamp: "Formatted at 2026-09-23 22:15:00 on dist-test-slave-l4q5"
server_key: "1b8a2862ab34ad5b1204c70ed4b064c1"
server_key_iv: "629c2ff4a67671d26201fdd07c49e1fb"
server_key_version: "encryptionkey@0"
I20260923 22:15:00.990862 15270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:01.002002 15270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:01.004292 15270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:01.004416 15270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:01.008052 15270 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:42437
I20260923 22:15:01.008102 15337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:42437 every 8 connection(s)
I20260923 22:15:01.008445 15270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:01.009436 15338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:01.011771 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap starting.
I20260923 22:15:01.012339 15338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:01.012580 15338 log.cc:824] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:01.013182 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15270
I20260923 22:15:01.013288 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:15:01.013432 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: No bootstrap required, opened a new log
I20260923 22:15:01.013471 1554 external_mini_cluster.cc:1428] Setting key 31a00248811e8771382eed24fe9a4eeb
I20260923 22:15:01.015791 15338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.015949 15338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:01.015980 15338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Initialized, Role: FOLLOWER
I20260923 22:15:01.016105 15338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.016213 15338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:01.016254 15338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:01.016301 15338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:01.016896 15338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.016999 15338 leader_election.cc:302] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e431d93dae4c85acb5e4569b5bcdef; no voters:
I20260923 22:15:01.017184 15338 leader_election.cc:288] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:15:01.017225 15343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:01.017403 15343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 LEADER]: Becoming Leader. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Running, Role: LEADER
I20260923 22:15:01.017457 15338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:01.017560 15343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.017904 15344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:01.017985 15344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:01.018146 15345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e431d93dae4c85acb5e4569b5bcdef. Latest consensus state: current_term: 1 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:01.018224 15345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:01.018354 15350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:01.018859 15350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:01.020713 15350 catalog_manager.cc:1347] Generated new cluster ID: 92df15bd11fa47bd9cff516b2260bf4c
I20260923 22:15:01.020767 15350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:01.024766 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.014022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.1.132.190:42437)
0923 22:15:01.014270 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.014279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.014322 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:15:01.014807 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.014813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.014862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.014995 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:01.015808 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.017091 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.018151 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.018826 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.018912 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.021622 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:01.021627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:01.021627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:01.021641 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:01.023276 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.023613 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.023617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.023617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.023704 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.023876 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.023877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.023877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.024131 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:01.024320 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.024411 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.024549 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260923 22:15:01.041492 15350 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:15:01.042127 15350 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:01.046160 15350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Generated new TSK 0
I20260923 22:15:01.046346 15350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:15:01.078825 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:01.160977 15364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:01.161224 15364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:01.161259 15364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:01.162537 15364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:01.162583 15364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:01.162622 15364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:15:01.164124 15364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:01.164381 15364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:01.164590 15364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:01.169379 15372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:01.169510 15364 server_base.cc:1034] running on GCE node
W20260923 22:15:01.169402 15369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.169376 15370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:01.169814 15364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:01.170039 15364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:01.171197 15364 hybrid_clock.cc:648] HybridClock initialized: now 1790201701171192 us; error 38 us; skew 500 ppm
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:01.173931 15364 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:15:01.174367 15364 webserver.cc:460] Webserver started at http://127.1.132.129:34851/ using document root <none> and password file <none>
I20260923 22:15:01.174564 15364 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:01.174639 15364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:01.174764 15364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:01.175802 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "7da7e434cd51b76f9ccc74875befb333"
server_key_iv: "d3046e74e942258a5caaa2c368ac9aaa"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.176123 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "7da7e434cd51b76f9ccc74875befb333"
server_key_iv: "d3046e74e942258a5caaa2c368ac9aaa"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.177455 15364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:01.178174 15379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.178324 15364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:15:01.178416 15364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "7da7e434cd51b76f9ccc74875befb333"
server_key_iv: "d3046e74e942258a5caaa2c368ac9aaa"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.178479 15364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:01.192770 15364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:01.195278 15364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:01.195418 15364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:01.195670 15364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:01.196009 15364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:01.196044 15364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.196084 15364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:01.196105 15364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.201777 15364 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:42073
I20260923 22:15:01.201850 15492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:42073 every 8 connection(s)
I20260923 22:15:01.202170 15364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:01.204066 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15364
I20260923 22:15:01.204170 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:15:01.204437 1554 external_mini_cluster.cc:1428] Setting key 578dce1ee77b9d45b6e65ead71c59919
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:15:01.215070 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.203372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:60905 (local address 127.1.132.190:42437)
0923 22:15:01.203431 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.203433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.203692 (+ 259us) server_negotiation.cc:407] Connection header received
0923 22:15:01.204533 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.204535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.204549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.204635 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:01.206017 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.206952 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.208652 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.209062 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.209159 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.211969 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:01.211971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:01.211971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:01.211980 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:01.213558 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.214034 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.214036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.214036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.214100 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.214385 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.214386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.214386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.214521 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:01.214583 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.214868 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.214993 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:15:01.215188 15495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.203296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:60905)
0923 22:15:01.203509 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.203556 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.204340 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.204785 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.204790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.205598 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:01.205905 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.205912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.207067 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.207068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.208539 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.208543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.209263 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.209265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.209444 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.210136 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:01.210147 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:01.211855 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:01.213657 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.213665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.213667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.213909 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.214184 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.214185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.214185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.214299 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.214600 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:01.214607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:01.214753 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.214848 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.214985 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260923 22:15:01.215802 15493 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
I20260923 22:15:01.215926 15493 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:01.216130 15493 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
I20260923 22:15:01.216672 15302 ts_manager.cc:194] Registered new tserver with Master: 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073)
I20260923 22:15:01.217335 15302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:60905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:15:01.233034 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:01.313748 15498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:01.313959 15498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:01.313983 15498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:01.315147 15498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:01.315192 15498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:01.315218 15498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:15:01.316723 15498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:01.316954 15498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:01.317171 15498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:01.322253 15503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.322240 15506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.322240 15504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:01.322501 15498 server_base.cc:1034] running on GCE node
I20260923 22:15:01.322710 15498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:01.322940 15498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:01.324080 15498 hybrid_clock.cc:648] HybridClock initialized: now 1790201701324069 us; error 28 us; skew 500 ppm
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:01.326491 15498 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:15:01.326900 15498 webserver.cc:460] Webserver started at http://127.1.132.130:42115/ using document root <none> and password file <none>
I20260923 22:15:01.327100 15498 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:01.327147 15498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:01.327286 15498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:01.328382 15498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "482d0242ef0e43efa20df2b1131db29a"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "016f11fbcdfbcaad871a71c4500c5d75"
server_key_iv: "894d2c028a636d019e81135a82ef1c27"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.328698 15498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "482d0242ef0e43efa20df2b1131db29a"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "016f11fbcdfbcaad871a71c4500c5d75"
server_key_iv: "894d2c028a636d019e81135a82ef1c27"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.329919 15498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:01.330660 15513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.330806 15498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:01.330883 15498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "482d0242ef0e43efa20df2b1131db29a"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "016f11fbcdfbcaad871a71c4500c5d75"
server_key_iv: "894d2c028a636d019e81135a82ef1c27"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.330941 15498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:01.356722 15498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:01.358793 15498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:01.358929 15498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:01.359153 15498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:01.359438 15498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:01.359470 15498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.359499 15498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:01.359516 15498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.364852 15498 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:42899
I20260923 22:15:01.364912 15626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:42899 every 8 connection(s)
I20260923 22:15:01.365221 15498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:15:01.368005 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15498
I20260923 22:15:01.368084 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:15:01.368268 1554 external_mini_cluster.cc:1428] Setting key 2b453bd1e7d1e087ad305bee7a26775f
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:01.375149 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.366052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:57423 (local address 127.1.132.190:42437)
0923 22:15:01.366100 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.366103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.366297 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:15:01.366815 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.366817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.366830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.366880 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:01.367834 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.368494 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.369638 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.369841 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.369896 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.372386 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:01.372388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:01.372388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:01.372396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:01.373669 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.374146 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.374147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.374147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.374194 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.374470 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.374472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.374472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.374598 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:01.374702 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.375012 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.375079 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:15:01.375401 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.366003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:57423)
0923 22:15:01.366178 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.366215 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.366716 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.366962 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.366966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.367538 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:01.367738 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.367742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.368598 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.368599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.369554 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.369558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.369916 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.369917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.370079 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.370684 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:01.370695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:01.372296 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:01.373754 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.373762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.373765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.374068 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.374250 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.374251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.374251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.374383 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.374692 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:01.374698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:01.374867 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.374980 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.375162 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260923 22:15:01.375872 15627 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
I20260923 22:15:01.375965 15627 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:01.376142 15627 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
I20260923 22:15:01.376534 15302 ts_manager.cc:194] Registered new tserver with Master: 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899)
I20260923 22:15:01.376909 15302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:57423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:15:01.395445 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:01.476292 15632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:01.476500 15632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:01.476521 15632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:01.477646 15632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:01.477680 15632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:01.477705 15632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:15:01.479174 15632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:01.479375 15632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:01.479561 15632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:01.484413 15638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.484514 15640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.484414 15637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:01.484545 15632 server_base.cc:1034] running on GCE node
I20260923 22:15:01.484861 15632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:01.485093 15632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:01.486233 15632 hybrid_clock.cc:648] HybridClock initialized: now 1790201701486218 us; error 31 us; skew 500 ppm
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:01.488616 15632 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:15:01.489076 15632 webserver.cc:460] Webserver started at http://127.1.132.131:36799/ using document root <none> and password file <none>
I20260923 22:15:01.489333 15632 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:01.489391 15632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:01.489521 15632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:01.490599 15632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "71c72288c5e743e999537b38cf3a9e4d"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "f78d37742639cc2ea00a775e58eba5d7"
server_key_iv: "82616ef0743374ca090fc336eee7bd56"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.490929 15632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "71c72288c5e743e999537b38cf3a9e4d"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "f78d37742639cc2ea00a775e58eba5d7"
server_key_iv: "82616ef0743374ca090fc336eee7bd56"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.492235 15632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:15:01.492977 15647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.493142 15632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:01.493216 15632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "71c72288c5e743e999537b38cf3a9e4d"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "f78d37742639cc2ea00a775e58eba5d7"
server_key_iv: "82616ef0743374ca090fc336eee7bd56"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.493283 15632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:01.506778 15632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:01.509065 15632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:01.509207 15632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:01.509434 15632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:01.509733 15632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:01.509766 15632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.509796 15632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:01.509812 15632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.515389 15632 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:38745
I20260923 22:15:01.515455 15760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:38745 every 8 connection(s)
I20260923 22:15:01.515738 15632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:15:01.520347 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15632
I20260923 22:15:01.520467 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:15:01.520618 1554 external_mini_cluster.cc:1428] Setting key dda71d5e0c13e6048a205d7472c18ffd
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:01.525759 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.516526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:52645 (local address 127.1.132.190:42437)
0923 22:15:01.516578 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.516581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.516777 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:15:01.517303 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.517304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.517317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.517375 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:01.518384 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.519164 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.520273 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.520511 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.520573 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.522886 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:01.522888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:01.522888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:01.522897 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:01.524462 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.524849 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.524851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.524851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.524910 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.525128 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.525128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.525129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.525250 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:01.525306 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.525580 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.525662 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:15:01.525952 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.516502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:52645)
0923 22:15:01.516660 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.516695 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.517204 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.517447 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.517450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.518089 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:01.518286 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.518291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.519264 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.519265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.520141 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.520144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.520583 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.520584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.520730 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.521306 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:01.521316 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:01.522796 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:01.524532 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.524540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.524543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.524785 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.524961 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:01.524962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:01.524962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:01.525073 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:01.525303 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:01.525309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:01.525453 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.525554 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.525726 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260923 22:15:01.526367 15761 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
I20260923 22:15:01.526532 15761 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:01.526737 15761 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
I20260923 22:15:01.527118 15302 ts_manager.cc:194] Registered new tserver with Master: 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
I20260923 22:15:01.527496 15302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:52645
I20260923 22:15:01.532496 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:15:01.539119 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.533781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.1.132.190:42437)
0923 22:15:01.533824 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.533827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.533947 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:15:01.534006 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.534007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.534016 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.534061 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:01.534611 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.535111 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.535816 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.536006 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.536059 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.537159 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:01.537160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:01.537160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:01.537166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:01.538222 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.538534 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.538536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.538536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.538581 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:01.538705 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:01.538705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:01.538705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:01.538838 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:01.538907 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.538980 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.539042 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:15:01.540723 15302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:01.542280 15302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:01.548869 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.1.132.130:42899)
0923 22:15:01.544788 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.544793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.544804 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:15:01.544842 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.544844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.544861 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.544940 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:01.545120 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.546253 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.547747 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.548501 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.548557 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.548619 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.548746 (+ 127us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.548785 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":87,"threads_started":1}
I20260923 22:15:01.548861 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.0.0.1:46908)
0923 22:15:01.544631 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.544640 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.544679 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.544974 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.544977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.544980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:01.545056 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.545058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.546345 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.546345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.547662 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.547665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.548573 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.548573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.548652 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.548665 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.548725 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.548782 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":9,"thread_start_us":57,"threads_started":1}
I20260923 22:15:01.549435 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:38745 (local address 127.0.0.1:40836)
0923 22:15:01.544665 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.544675 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.544728 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.544994 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.544995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.544996 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:01.545056 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.545058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.547263 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.547265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.548339 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.548342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.549216 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.549217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.549274 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.549279 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.549333 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.549373 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":35,"threads_started":1}
I20260923 22:15:01.549471 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.1.132.131:38745)
0923 22:15:01.544856 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.544860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.544868 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:01.544904 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.544906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.544920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.544998 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:01.546374 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.547158 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.548425 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.549136 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.549210 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.549260 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.549347 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.549391 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260923 22:15:01.549644 15769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.0.0.1:33824)
0923 22:15:01.544498 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.544538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.544602 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.545220 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.545221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.545223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:01.545258 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.545259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.546511 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.546512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.547663 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.547665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.549435 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.549436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.549504 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.549508 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.549538 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.549580 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":9,"thread_start_us":53,"threads_started":1}
I20260923 22:15:01.549764 15561 tablet_service.cc:1467] Processing CreateTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 (DEFAULT_TABLE table=test-table [id=e2c4203aace646be95d6b586bab0ccb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:01.550117 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.544453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.1.132.129:42073)
0923 22:15:01.545077 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.545080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.545089 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:15:01.545124 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.545126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.545139 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.545217 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:01.545311 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.546389 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.548266 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.549336 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.549895 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.549990 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.550015 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.550054 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":103,"threads_started":1}
I20260923 22:15:01.550155 15561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdba58b6c1e748b686d22ea63274b6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:01.550113 15695 tablet_service.cc:1467] Processing CreateTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 (DEFAULT_TABLE table=test-table [id=e2c4203aace646be95d6b586bab0ccb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:01.550422 15695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdba58b6c1e748b686d22ea63274b6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:01.552860 15427 tablet_service.cc:1467] Processing CreateTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 (DEFAULT_TABLE table=test-table [id=e2c4203aace646be95d6b586bab0ccb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:01.553172 15427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdba58b6c1e748b686d22ea63274b6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:01.553855 15776 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Bootstrap starting.
I20260923 22:15:01.554597 15776 tablet_bootstrap.cc:654] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:01.555189 15776 log.cc:824] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:01.555598 15775 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Bootstrap starting.
I20260923 22:15:01.556305 15775 tablet_bootstrap.cc:654] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:01.556614 15778 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap starting.
I20260923 22:15:01.556560 15776 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: No bootstrap required, opened a new log
I20260923 22:15:01.556761 15776 ts_tablet_manager.cc:1397] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:15:01.556845 15775 log.cc:824] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:01.557233 15778 tablet_bootstrap.cc:654] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:01.557775 15775 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: No bootstrap required, opened a new log
I20260923 22:15:01.557780 15778 log.cc:824] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:01.557853 15775 ts_tablet_manager.cc:1397] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:01.558570 15778 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: No bootstrap required, opened a new log
I20260923 22:15:01.558641 15778 ts_tablet_manager.cc:1397] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:15:01.559090 15776 raft_consensus.cc:348] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.559239 15776 raft_consensus.cc:374] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:01.559271 15776 raft_consensus.cc:729] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482d0242ef0e43efa20df2b1131db29a, State: Initialized, Role: FOLLOWER
I20260923 22:15:01.559392 15776 consensus_queue.cc:260] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.559706 15776 ts_tablet_manager.cc:1428] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:15:01.559734 15627 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
I20260923 22:15:01.560137 15775 raft_consensus.cc:348] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.560364 15775 raft_consensus.cc:374] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:01.560400 15775 raft_consensus.cc:729] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c72288c5e743e999537b38cf3a9e4d, State: Initialized, Role: FOLLOWER
I20260923 22:15:01.560526 15775 consensus_queue.cc:260] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.560575 15778 raft_consensus.cc:348] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.560722 15778 raft_consensus.cc:374] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:01.560753 15778 raft_consensus.cc:729] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581d3a385f7f49f99ce2e13a3f1ef4bc, State: Initialized, Role: FOLLOWER
I20260923 22:15:01.560796 15775 ts_tablet_manager.cc:1428] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:15:01.560904 15761 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
I20260923 22:15:01.560882 15778 consensus_queue.cc:260] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.561124 15778 ts_tablet_manager.cc:1428] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:15:01.561208 15493 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
W20260923 22:15:01.615795 15628 tablet.cc:2369] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:15:01.703013 15494 tablet.cc:2369] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:01.705371 15782 raft_consensus.cc:482] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:01.705531 15782 raft_consensus.cc:504] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.705875 15782 leader_election.cc:288] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073), 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899)
I20260923 22:15:01.709182 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.706212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:34405 (local address 127.1.132.129:42073)
0923 22:15:01.706267 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.706269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.706349 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:15:01.706387 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.706388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.706399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.706476 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:01.706811 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.707305 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.708366 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.708828 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.708889 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.708897 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.709064 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.709108 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:15:01.709172 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.706089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.1.132.131:34405)
0923 22:15:01.706141 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.706154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.706201 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.706646 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.706647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.706649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:01.706721 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.706722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.707393 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.707394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.708266 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.708268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.708913 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.708914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.708993 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.708998 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.709050 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.709095 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1}
I20260923 22:15:01.709268 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.706142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:36147 (local address 127.1.132.130:42899)
0923 22:15:01.706193 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.706196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.706493 (+ 297us) server_negotiation.cc:407] Connection header received
0923 22:15:01.706537 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.706538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.706546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.706599 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:01.706790 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.707301 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.708507 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.708910 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.709014 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.709025 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.709172 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.709211 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:15:01.709475 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.706094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.1.132.131:36147)
0923 22:15:01.706391 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.706402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:01.706462 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:01.706632 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:01.706633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:01.706635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:01.706709 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.706711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.707381 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.707387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.708297 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:01.708299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.709014 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.709015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:01.709097 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.709101 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:01.709155 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:15:01.709196 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":98,"thread_start_us":120,"threads_started":1}
I20260923 22:15:01.709633 15447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdba58b6c1e748b686d22ea63274b6b9" candidate_uuid: "71c72288c5e743e999537b38cf3a9e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" is_pre_election: true
I20260923 22:15:01.709803 15447 raft_consensus.cc:2454] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71c72288c5e743e999537b38cf3a9e4d in term 0.
I20260923 22:15:01.709923 15581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdba58b6c1e748b686d22ea63274b6b9" candidate_uuid: "71c72288c5e743e999537b38cf3a9e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482d0242ef0e43efa20df2b1131db29a" is_pre_election: true
I20260923 22:15:01.710049 15649 leader_election.cc:302] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc, 71c72288c5e743e999537b38cf3a9e4d; no voters:
I20260923 22:15:01.710094 15581 raft_consensus.cc:2454] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71c72288c5e743e999537b38cf3a9e4d in term 0.
I20260923 22:15:01.710187 15782 raft_consensus.cc:2781] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:01.710278 15782 raft_consensus.cc:482] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:01.710311 15782 raft_consensus.cc:3037] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:01.711036 15782 raft_consensus.cc:504] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.711163 15782 leader_election.cc:288] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [CANDIDATE]: Term 1 election: Requested vote from peers 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073), 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899)
I20260923 22:15:01.711366 15581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdba58b6c1e748b686d22ea63274b6b9" candidate_uuid: "71c72288c5e743e999537b38cf3a9e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482d0242ef0e43efa20df2b1131db29a"
I20260923 22:15:01.711444 15447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdba58b6c1e748b686d22ea63274b6b9" candidate_uuid: "71c72288c5e743e999537b38cf3a9e4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
I20260923 22:15:01.711458 15581 raft_consensus.cc:3037] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:01.711531 15447 raft_consensus.cc:3037] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:01.712443 15447 raft_consensus.cc:2454] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71c72288c5e743e999537b38cf3a9e4d in term 1.
I20260923 22:15:01.712641 15581 raft_consensus.cc:2454] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71c72288c5e743e999537b38cf3a9e4d in term 1.
I20260923 22:15:01.712731 15649 leader_election.cc:302] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc, 71c72288c5e743e999537b38cf3a9e4d; no voters:
I20260923 22:15:01.712824 15782 raft_consensus.cc:2781] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:01.712978 15782 raft_consensus.cc:686] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 1 LEADER]: Becoming Leader. State: Replica: 71c72288c5e743e999537b38cf3a9e4d, State: Running, Role: LEADER
I20260923 22:15:01.713073 15782 consensus_queue.cc:237] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:01.714037 15301 catalog_manager.cc:5495] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131). New cstate: current_term: 1 leader_uuid: "71c72288c5e743e999537b38cf3a9e4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } health_report { overall_health: HEALTHY } } }
I20260923 22:15:01.734762 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.732286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.1.132.131:38745)
0923 22:15:01.732340 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:01.732342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:01.732349 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:15:01.732424 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:01.732425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:01.732444 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:01.732505 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:01.732634 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.733208 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.734015 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:01.734252 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:01.734312 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:01.734424 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:01.734541 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:01.734609 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:01.734641 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:15:01.734662 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:15:01.736549 15447 raft_consensus.cc:1261] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Refusing update from remote peer 71c72288c5e743e999537b38cf3a9e4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:01.736549 15581 raft_consensus.cc:1261] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Refusing update from remote peer 71c72288c5e743e999537b38cf3a9e4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:01.736881 15785 consensus_queue.cc:1035] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:01.737032 15782 consensus_queue.cc:1035] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:01.739219 15788 mvcc.cc:205] Tried to move back new op lower bound from 7332666170310144000 to 7332666170217472000. Current Snapshot: MvccSnapshot[applied={T|T < 7332666170310144000}]
I20260923 22:15:01.739670 15789 mvcc.cc:205] Tried to move back new op lower bound from 7332666170310144000 to 7332666170217472000. Current Snapshot: MvccSnapshot[applied={T|T < 7332666170310144000}]
I20260923 22:15:01.741693 15301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:01.741791 15301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:01.742574 15301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Sending DeleteTablet for 3 replicas of tablet bdba58b6c1e748b686d22ea63274b6b9
I20260923 22:15:01.742921 15427 tablet_service.cc:1514] Processing DeleteTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:01 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:33824
I20260923 22:15:01.743189 15798 tablet_replica.cc:330] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: stopping tablet replica
I20260923 22:15:01.743322 15561 tablet_service.cc:1514] Processing DeleteTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:01 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:46908
I20260923 22:15:01.743366 15695 tablet_service.cc:1514] Processing DeleteTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:01 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:40836
I20260923 22:15:01.743444 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15364
I20260923 22:15:01.744112 15800 tablet_replica.cc:330] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: stopping tablet replica
I20260923 22:15:01.744243 15800 raft_consensus.cc:2229] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:15:01.744349 15800 raft_consensus.cc:2258] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:01.744761 15800 ts_tablet_manager.cc:1905] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:01.746201 15800 ts_tablet_manager.cc:1918] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:01.746264 15800 log.cc:1196] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/bdba58b6c1e748b686d22ea63274b6b9
I20260923 22:15:01.746506 15800 ts_tablet_manager.cc:1939] T bdba58b6c1e748b686d22ea63274b6b9 P 71c72288c5e743e999537b38cf3a9e4d: Deleting consensus metadata
I20260923 22:15:01.746886 15286 catalog_manager.cc:4841] TS 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): tablet bdba58b6c1e748b686d22ea63274b6b9 (table test-table [id=e2c4203aace646be95d6b586bab0ccb0]) successfully deleted
I20260923 22:15:01.748307 15799 tablet_replica.cc:330] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: stopping tablet replica
I20260923 22:15:01.748430 15799 raft_consensus.cc:2229] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:01.748549 15799 raft_consensus.cc:2258] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:01.748939 15799 ts_tablet_manager.cc:1905] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:15:01.750308 15287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:42073 (error 108)
I20260923 22:15:01.750419 15799 ts_tablet_manager.cc:1918] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:01.750494 15799 log.cc:1196] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/bdba58b6c1e748b686d22ea63274b6b9
I20260923 22:15:01.750756 15799 ts_tablet_manager.cc:1939] T bdba58b6c1e748b686d22ea63274b6b9 P 482d0242ef0e43efa20df2b1131db29a: Deleting consensus metadata
I20260923 22:15:01.750854 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15498
I20260923 22:15:01.751363 15769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.750759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.0.0.1:33836)
0923 22:15:01.751286 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.751329 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:42073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503}
W20260923 22:15:01.751483 15287 catalog_manager.cc:4584] TS 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bdba58b6c1e748b686d22ea63274b6b9: Network error: Client connection negotiation failed: client connection to 127.1.132.129:42073: connect: Connection refused (error 111)
I20260923 22:15:01.756253 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15632
I20260923 22:15:01.756322 15769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:01.756204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.0.0.1:46916)
0923 22:15:01.756251 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:01.756283 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.130:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260923 22:15:01.761209 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15270
I20260923 22:15:01.766327 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--webserver_interface=127.1.132.190
--webserver_port=45175
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:01.849262 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:01.849475 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:01.849498 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:01.850661 15802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:01.850700 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:01.850713 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:01.850724 15802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:01.850735 15802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:01.852149 15802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:01.852420 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:01.852636 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:01.860869 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.861215 15811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:01.861215 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:01.861413 15802 server_base.cc:1034] running on GCE node
I20260923 22:15:01.861641 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:01.861917 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:01.863070 15802 hybrid_clock.cc:648] HybridClock initialized: now 1790201701863049 us; error 33 us; skew 500 ppm
Sep 23 22:15:01 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:01.865588 15802 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:15:01.866058 15802 webserver.cc:460] Webserver started at http://127.1.132.190:45175/ using document root <none> and password file <none>
I20260923 22:15:01.866284 15802 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:01.866348 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:01.867700 15802 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:01.868386 15818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:01.868541 15802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:01.868628 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "16e431d93dae4c85acb5e4569b5bcdef"
format_stamp: "Formatted at 2026-09-23 22:15:00 on dist-test-slave-l4q5"
server_key: "1b8a2862ab34ad5b1204c70ed4b064c1"
server_key_iv: "629c2ff4a67671d26201fdd07c49e1fb"
server_key_version: "encryptionkey@0"
I20260923 22:15:01.869015 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:01.900564 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:01.902563 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:01.902704 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:01.906575 15802 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:42437
I20260923 22:15:01.906620 15870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:42437 every 8 connection(s)
I20260923 22:15:01.906994 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:01.908751 15871 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:15:01.909451 15871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap starting.
I20260923 22:15:01.911657 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15802
I20260923 22:15:01.911892 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:42073
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=34851
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:01.917563 15871 log.cc:824] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:01.919617 15871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:01.919909 15871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap complete.
I20260923 22:15:01.922099 15871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.922334 15871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Initialized, Role: FOLLOWER
I20260923 22:15:01.922483 15871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.922547 15871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:01.922582 15871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:01.922627 15871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:15:01.923655 15871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.923749 15871 leader_election.cc:302] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e431d93dae4c85acb5e4569b5bcdef; no voters:
I20260923 22:15:01.923923 15871 leader_election.cc:288] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:15:01.923972 15875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:15:01.924122 15875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 LEADER]: Becoming Leader. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Running, Role: LEADER
I20260923 22:15:01.924222 15871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:01.924310 15875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:01.924642 15875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e431d93dae4c85acb5e4569b5bcdef. Latest consensus state: current_term: 2 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:01.924715 15876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:01.924721 15875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:01.924789 15876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:01.924953 15879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:01.925784 15879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:01.925894 15879 catalog_manager.cc:1259] Loaded cluster ID: 92df15bd11fa47bd9cff516b2260bf4c
I20260923 22:15:01.925940 15879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:01.926281 15879 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 22:15:01.929018 15879 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 22:15:01.930846 15879 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:01.931057 15879 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Loaded TSK: 0
I20260923 22:15:01.931267 15879 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:15:02.008510 15873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:02.008734 15873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:02.008770 15873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:02.009994 15873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:02.010037 15873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:02.010075 15873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:15:02.011570 15873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:42073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=34851
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:02.011818 15873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:02.012042 15873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:02.017088 15898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.017234 15873 server_base.cc:1034] running on GCE node
W20260923 22:15:02.017110 15899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.017391 15901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.017699 15873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:02.018009 15873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:02.019196 15873 hybrid_clock.cc:648] HybridClock initialized: now 1790201702019182 us; error 28 us; skew 500 ppm
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:02.025883 15873 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:15:02.026424 15873 webserver.cc:460] Webserver started at http://127.1.132.129:34851/ using document root <none> and password file <none>
I20260923 22:15:02.026669 15873 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:02.026727 15873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:02.028172 15873 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:02.028961 15908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.029160 15873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:02.029259 15873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "7da7e434cd51b76f9ccc74875befb333"
server_key_iv: "d3046e74e942258a5caaa2c368ac9aaa"
server_key_version: "encryptionkey@0"
I20260923 22:15:02.029726 15873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:02.050822 15873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:02.053470 15873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:02.053635 15873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:02.053907 15873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:02.054411 15915 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:15:02.055910 15873 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:15:02.055972 15873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:15:02.056007 15873 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:15:02.056927 15873 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:15:02.056977 15873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:02.057044 15915 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap starting.
I20260923 22:15:02.064013 15873 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:42073
I20260923 22:15:02.064292 16022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:42073 every 8 connection(s)
I20260923 22:15:02.064536 15873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:02.067167 15915 log.cc:824] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:02.067193 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 15873
I20260923 22:15:02.067534 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:42899
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=42115
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:02.070489 15915 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:02.070820 15915 tablet_bootstrap.cc:492] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap complete.
I20260923 22:15:02.072013 15915 ts_tablet_manager.cc:1397] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.010s
I20260923 22:15:02.073495 15915 raft_consensus.cc:348] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:02.073815 15915 raft_consensus.cc:729] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581d3a385f7f49f99ce2e13a3f1ef4bc, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.073968 15915 consensus_queue.cc:260] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } } peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } }
I20260923 22:15:02.074257 15915 ts_tablet_manager.cc:1428] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:02.079550 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.065266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:44371)
0923 22:15:02.067485 (+ 2219us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.067536 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.069386 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.070026 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.070031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.070378 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:02.070660 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.070668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.072288 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.072289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.073504 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.073507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.074384 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.074385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.074480 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.075002 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:02.075018 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:02.076359 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:02.078049 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.078057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.078059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.078277 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.078514 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.078515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.078515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.078622 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.078953 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:02.078959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:02.079095 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.079181 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.079352 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2173,"thread_start_us":109,"threads_started":1}
I20260923 22:15:02.079550 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.065371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:44371 (local address 127.1.132.190:42437)
0923 22:15:02.069086 (+ 3715us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.069094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.069130 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:15:02.069687 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.069693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.069757 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.069884 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:02.070800 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.072139 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.073611 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.074192 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.074293 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.076459 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:02.076462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:02.076462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:02.076473 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:02.077969 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.078352 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.078366 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.078366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.078453 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.078676 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.078677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.078677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.078893 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:02.078963 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.079242 (+ 279us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.079362 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3662,"thread_start_us":86,"threads_started":1}
I20260923 22:15:02.080801 16023 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
I20260923 22:15:02.080895 16023 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:02.081061 16023 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
I20260923 22:15:02.081722 15835 ts_manager.cc:194] Registered new tserver with Master: 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073)
I20260923 22:15:02.082772 15835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:44371
I20260923 22:15:02.083738 16023 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201701, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:15:02.091893 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.082347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.1.132.129:42073)
0923 22:15:02.082510 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.082515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.082551 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:15:02.082699 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.082702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.082715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.082803 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:02.083294 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.084130 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.085400 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.085869 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.085932 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.088444 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:02.088454 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:02.088454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:02.088463 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:02.090172 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.090988 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.090991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.090992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.091060 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.091260 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.091261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.091261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.091430 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:02.091502 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.091692 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.091787 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260923 22:15:02.091902 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.082267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.0.0.1:33840)
0923 22:15:02.082465 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.082495 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.082617 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.082812 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.082816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.083158 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:02.083217 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.083220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.084239 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.084240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.085283 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.085288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.085961 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.085962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.086052 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.086529 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:02.086535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:02.088272 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:02.090439 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.090451 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.090454 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.090872 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.091134 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.091134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.091135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.091207 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.091493 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:02.091502 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:02.091568 (+ 66us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.091656 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.091800 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260923 22:15:02.092590 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet bdba58b6c1e748b686d22ea63274b6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:01 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:33840
I20260923 22:15:02.093133 16032 tablet_replica.cc:330] stopping tablet replica
I20260923 22:15:02.093230 16032 raft_consensus.cc:2229] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:02.093299 16032 raft_consensus.cc:2258] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:02.093608 16032 ts_tablet_manager.cc:1905] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:02.095485 16032 ts_tablet_manager.cc:1918] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:02.095551 16032 log.cc:1196] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/bdba58b6c1e748b686d22ea63274b6b9
I20260923 22:15:02.095770 16032 ts_tablet_manager.cc:1939] T bdba58b6c1e748b686d22ea63274b6b9 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Deleting consensus metadata
I20260923 22:15:02.096251 15820 catalog_manager.cc:4841] TS 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073): tablet bdba58b6c1e748b686d22ea63274b6b9 (table test-table [id=e2c4203aace646be95d6b586bab0ccb0]) successfully deleted
W20260923 22:15:02.156589 16028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:02.156814 16028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:02.156848 16028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:02.158114 16028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:02.158169 16028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:02.158221 16028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:15:02.160354 16028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:42899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=42115
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:02.160657 16028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:02.160885 16028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:02.166316 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.166329 16038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.167011 16037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.167146 16028 server_base.cc:1034] running on GCE node
I20260923 22:15:02.167378 16028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:02.167640 16028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:02.168816 16028 hybrid_clock.cc:648] HybridClock initialized: now 1790201702168803 us; error 30 us; skew 500 ppm
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:02.171325 16028 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:15:02.171837 16028 webserver.cc:460] Webserver started at http://127.1.132.130:42115/ using document root <none> and password file <none>
I20260923 22:15:02.172062 16028 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:02.172124 16028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:02.173542 16028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:02.174153 16047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.174302 16028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:02.174387 16028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "482d0242ef0e43efa20df2b1131db29a"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "016f11fbcdfbcaad871a71c4500c5d75"
server_key_iv: "894d2c028a636d019e81135a82ef1c27"
server_key_version: "encryptionkey@0"
I20260923 22:15:02.174741 16028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:02.198426 16028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:02.200428 16028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:02.200567 16028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:02.200806 16028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:02.201125 16028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:02.201159 16028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.201188 16028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:02.201210 16028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.206821 16028 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:42899
I20260923 22:15:02.206866 16160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:42899 every 8 connection(s)
I20260923 22:15:02.207211 16028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:02.214500 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16028
I20260923 22:15:02.214763 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:38745
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=36799
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:02.217065 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.207945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:51985 (local address 127.1.132.190:42437)
0923 22:15:02.208035 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.208037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.208233 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:15:02.208768 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.208769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.208781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.208843 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:02.209844 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.210371 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.211431 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.211664 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.211737 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.214001 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:02.214003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:02.214003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:02.214011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:02.215394 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.215996 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.215998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.215999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.216060 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.216354 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.216355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.216356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.216499 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:02.216561 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.216859 (+ 298us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.216969 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 22:15:02.217228 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.207928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:51985)
0923 22:15:02.208088 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.208131 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.208649 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.208902 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.208906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.209493 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:02.209712 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.209718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.210465 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.210466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.211324 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.211327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.211749 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.211749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.211863 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.212460 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:02.212469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:02.213898 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:02.215487 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.215497 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.215500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.215877 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.216121 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.216121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.216122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.216294 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.216559 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:02.216567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:02.216719 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.216812 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.216995 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260923 22:15:02.217905 16161 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
I20260923 22:15:02.218026 16161 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:02.218236 16161 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
I20260923 22:15:02.218725 15835 ts_manager.cc:194] Registered new tserver with Master: 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899)
I20260923 22:15:02.219126 15835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:51985
W20260923 22:15:02.297158 16164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:02.297370 16164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:02.297394 16164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:02.298704 16164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:02.298763 16164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:02.298794 16164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:15:02.300659 16164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:38745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=36799
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:02.300927 16164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:02.301173 16164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:02.306290 16169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.306250 16172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.306300 16170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.306465 16164 server_base.cc:1034] running on GCE node
I20260923 22:15:02.306701 16164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:02.306941 16164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:02.308105 16164 hybrid_clock.cc:648] HybridClock initialized: now 1790201702308091 us; error 27 us; skew 500 ppm
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:02.310765 16164 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:15:02.311216 16164 webserver.cc:460] Webserver started at http://127.1.132.131:36799/ using document root <none> and password file <none>
I20260923 22:15:02.311419 16164 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:02.311470 16164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:02.312989 16164 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:02.313663 16179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.313863 16164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:02.313959 16164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "71c72288c5e743e999537b38cf3a9e4d"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "f78d37742639cc2ea00a775e58eba5d7"
server_key_iv: "82616ef0743374ca090fc336eee7bd56"
server_key_version: "encryptionkey@0"
I20260923 22:15:02.314327 16164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:02.329864 16164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:02.331960 16164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:02.332113 16164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:02.332378 16164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:02.332680 16164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:02.332711 16164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.332741 16164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:02.332762 16164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.338347 16164 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:38745
I20260923 22:15:02.338400 16292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:38745 every 8 connection(s)
I20260923 22:15:02.338747 16164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:15:02.340372 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16164
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:02.347584 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.340750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.1.132.190:42437)
0923 22:15:02.340849 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.340851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.340859 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:02.340899 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.340900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.340908 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.340956 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:02.341547 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.342306 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.343383 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.343580 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.343627 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.345018 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:02.345019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:02.345019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:02.345026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:02.346443 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.346800 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.346801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.346801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.346855 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.347046 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.347046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.347047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.347187 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:02.347305 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.347426 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.347490 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
I20260923 22:15:02.349004 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.339478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:50939 (local address 127.1.132.190:42437)
0923 22:15:02.339537 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.339539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.339700 (+ 161us) server_negotiation.cc:407] Connection header received
0923 22:15:02.340196 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.340197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.340209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.340277 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:02.341496 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.342300 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.343685 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.343856 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.343914 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.346222 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:02.346223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:02.346223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:02.346230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:02.347794 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.348173 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.348174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.348174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.348255 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:02.348535 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:02.348535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:02.348535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:02.348636 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:02.348690 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.348887 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.348946 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":32}
I20260923 22:15:02.349190 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.339415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:50939)
0923 22:15:02.339580 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.339623 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.340097 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.340354 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.340358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.341106 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:02.341405 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.341411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.342420 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.342421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.343607 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.343610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.343934 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.343934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.344046 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.344490 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:02.344499 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:02.346118 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:02.347870 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.347879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.347882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.348129 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.348345 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:02.348346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:02.348346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:02.348480 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:02.348685 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:02.348688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:02.348802 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.348876 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.348994 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260923 22:15:02.349572 16293 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42437
I20260923 22:15:02.349668 16293 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:02.349838 16293 heartbeater.cc:511] Master 127.1.132.190:42437 requested a full tablet report, sending...
I20260923 22:15:02.350175 15835 ts_manager.cc:194] Registered new tserver with Master: 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
I20260923 22:15:02.350524 15835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:50939
I20260923 22:15:02.359395 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:15:02.362213 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.359808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.1.132.190:42437)
0923 22:15:02.359895 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.359897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.359955 (+ 58us) server_negotiation.cc:407] Connection header received
0923 22:15:02.360015 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.360015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.360051 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.360119 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:02.360270 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.360757 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.361542 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.361729 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.361779 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.361907 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:02.362008 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:02.362070 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.362141 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.362155 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 22:15:02.362543 15835 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:02.364215 15835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:02.366349 15957 tablet_service.cc:1467] Processing CreateTablet for tablet 9e79c6a487b0461fbf695463124f3366 (DEFAULT_TABLE table=test-table [id=fbb8e5419ed849e79c361d78cb8d9e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:02.366570 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e79c6a487b0461fbf695463124f3366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:02.368063 15915 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap starting.
I20260923 22:15:02.368480 15915 tablet_bootstrap.cc:654] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:02.369079 15915 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: No bootstrap required, opened a new log
I20260923 22:15:02.369133 15915 ts_tablet_manager.cc:1397] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:02.369494 15915 raft_consensus.cc:348] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.369575 15915 raft_consensus.cc:374] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:02.369640 15915 raft_consensus.cc:729] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581d3a385f7f49f99ce2e13a3f1ef4bc, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.369688 15915 consensus_queue.cc:260] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.369778 15915 ts_tablet_manager.cc:1428] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:02.370580 16299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.366251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:38745 (local address 127.0.0.1:40848)
0923 22:15:02.366422 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.366433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.366493 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.367080 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.367081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.367084 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.367136 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.367138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.368243 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.368244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.369392 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.369395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.370344 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.370345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.370405 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.370425 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.370467 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.370507 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260923 22:15:02.370805 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.366683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.1.132.131:38745)
0923 22:15:02.366906 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.366910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.366919 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:15:02.366956 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.366957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.366971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.367040 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.367210 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.368021 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.369471 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.370185 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.370250 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.370312 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.370682 (+ 370us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.370732 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":150,"threads_started":1}
I20260923 22:15:02.371475 16227 tablet_service.cc:1467] Processing CreateTablet for tablet 9e79c6a487b0461fbf695463124f3366 (DEFAULT_TABLE table=test-table [id=fbb8e5419ed849e79c361d78cb8d9e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:02.371644 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.366163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.0.0.1:46930)
0923 22:15:02.366197 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.366208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.366242 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.368338 (+ 2096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.368340 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.368342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.368446 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.368448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.369594 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.369595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.370605 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.370608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.371385 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.371386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.371470 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.371475 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.371533 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.371584 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":64}
I20260923 22:15:02.371678 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.366428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.1.132.130:42899)
0923 22:15:02.368139 (+ 1711us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.368143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.368155 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:15:02.368220 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.368223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.368243 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.368337 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.368526 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.369497 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.370688 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.371308 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.371369 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.371406 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.371543 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.371596 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1669,"thread_start_us":78,"threads_started":1}
I20260923 22:15:02.371834 16227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e79c6a487b0461fbf695463124f3366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:02.372467 16095 tablet_service.cc:1467] Processing CreateTablet for tablet 9e79c6a487b0461fbf695463124f3366 (DEFAULT_TABLE table=test-table [id=fbb8e5419ed849e79c361d78cb8d9e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:02.372776 16095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e79c6a487b0461fbf695463124f3366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:02.375401 16303 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Bootstrap starting.
I20260923 22:15:02.375861 16304 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Bootstrap starting.
I20260923 22:15:02.375974 16303 tablet_bootstrap.cc:654] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:02.376408 16304 tablet_bootstrap.cc:654] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:02.376474 16303 log.cc:824] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:02.376904 16304 log.cc:824] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:02.377218 16303 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: No bootstrap required, opened a new log
I20260923 22:15:02.377290 16303 ts_tablet_manager.cc:1397] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:15:02.377602 16304 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: No bootstrap required, opened a new log
I20260923 22:15:02.377665 16304 ts_tablet_manager.cc:1397] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:15:02.378921 16303 raft_consensus.cc:348] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.379071 16303 raft_consensus.cc:374] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:02.379104 16303 raft_consensus.cc:729] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c72288c5e743e999537b38cf3a9e4d, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.379215 16303 consensus_queue.cc:260] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.379196 16304 raft_consensus.cc:348] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.379330 16304 raft_consensus.cc:374] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:02.379367 16304 raft_consensus.cc:729] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482d0242ef0e43efa20df2b1131db29a, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.379490 16303 ts_tablet_manager.cc:1428] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:02.379511 16293 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
I20260923 22:15:02.379496 16304 consensus_queue.cc:260] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.379772 16304 ts_tablet_manager.cc:1428] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:02.379799 16161 heartbeater.cc:503] Master 127.1.132.190:42437 was elected leader, sending a full tablet report...
W20260923 22:15:02.457823 16162 tablet.cc:2369] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:15:02.568753 16024 tablet.cc:2369] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:02.583262 16029 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:02.583405 16029 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.583688 16029 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
I20260923 22:15:02.586619 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.584157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:41437 (local address 127.1.132.131:38745)
0923 22:15:02.584229 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.584231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.584241 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:02.584293 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.584295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.584303 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.584357 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.584477 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.584974 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.585931 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.586298 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.586351 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.586359 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.586510 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.586547 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:15:02.586678 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.583942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:56347 (local address 127.1.132.130:42899)
0923 22:15:02.583994 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.583996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.584084 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:15:02.584241 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.584243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.584253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.584284 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.584539 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.584998 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.585921 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.586362 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.586428 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.586436 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.586581 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.586619 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:15:02.586633 16310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.584068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:38745 (local address 127.1.132.129:41437)
0923 22:15:02.584154 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.584164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.584247 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.584361 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.584362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.584363 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.584425 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.584426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.585049 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.585050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.585841 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.585843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.586368 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.586369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.586427 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.586453 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.586495 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.586529 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":8,"thread_start_us":37,"threads_started":1}
I20260923 22:15:02.586899 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.583891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.1.132.129:56347)
0923 22:15:02.584011 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.584022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.584100 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.584360 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.584360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.584363 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.584464 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.584466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.585093 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.585094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.585839 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.585841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.586457 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.586458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.586512 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.586517 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.586568 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.586605 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":5,"thread_start_us":68,"threads_started":1}
I20260923 22:15:02.586884 16308 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:02.586990 16308 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.587227 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482d0242ef0e43efa20df2b1131db29a" is_pre_election: true
I20260923 22:15:02.587322 16115 raft_consensus.cc:2454] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581d3a385f7f49f99ce2e13a3f1ef4bc in term 0.
I20260923 22:15:02.587278 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71c72288c5e743e999537b38cf3a9e4d" is_pre_election: true
I20260923 22:15:02.587354 16308 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745), 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073)
I20260923 22:15:02.587411 16247 raft_consensus.cc:2454] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581d3a385f7f49f99ce2e13a3f1ef4bc in term 0.
I20260923 22:15:02.587664 15909 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482d0242ef0e43efa20df2b1131db29a, 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters:
I20260923 22:15:02.587782 16029 raft_consensus.cc:2781] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:02.587845 16029 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:02.587877 16029 raft_consensus.cc:3037] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:02.588804 16029 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.588937 16029 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 1 election: Requested vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
I20260923 22:15:02.589130 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482d0242ef0e43efa20df2b1131db29a"
I20260923 22:15:02.589238 16115 raft_consensus.cc:3037] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 0 FOLLOWER]: Advancing to term 1
W20260923 22:15:02.589254 16294 tablet.cc:2369] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:02.590039 16115 raft_consensus.cc:2454] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581d3a385f7f49f99ce2e13a3f1ef4bc in term 1.
I20260923 22:15:02.590149 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71c72288c5e743e999537b38cf3a9e4d"
I20260923 22:15:02.590214 16247 raft_consensus.cc:3037] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:02.590224 15909 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482d0242ef0e43efa20df2b1131db29a, 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters:
I20260923 22:15:02.590305 16029 raft_consensus.cc:2781] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:02.590456 16029 raft_consensus.cc:686] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 LEADER]: Becoming Leader. State: Replica: 581d3a385f7f49f99ce2e13a3f1ef4bc, State: Running, Role: LEADER
I20260923 22:15:02.590526 16029 consensus_queue.cc:237] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.590714 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.587533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.1.132.130:38205)
0923 22:15:02.587577 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.587587 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.587622 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.587721 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.587722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.587723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.587781 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.587782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.588725 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.588726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.589755 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.589758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.590485 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.590486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.590563 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.590567 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.590615 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.590652 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:15:02.590713 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.587838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:52031 (local address 127.1.132.131:38745)
0923 22:15:02.588028 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.588030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.588034 (+ 4us) server_negotiation.cc:407] Connection header received
0923 22:15:02.588048 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.588049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.588063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.588101 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.588263 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.588750 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.589894 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.590358 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.590424 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.590433 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.590625 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.590657 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260923 22:15:02.590890 16311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.587674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:38745 (local address 127.1.132.130:52031)
0923 22:15:02.587907 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.587916 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.587962 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.588120 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:02.588121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:02.588122 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:02.588214 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.588216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.588833 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.588834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.589816 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:02.589819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.590463 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.590464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:02.590535 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.590539 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:02.590779 (+ 240us) client_negotiation.cc:240] Negotiation successful
0923 22:15:02.590819 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260923 22:15:02.590956 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.587584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:38205 (local address 127.1.132.129:42073)
0923 22:15:02.587624 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.587626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.587633 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:15:02.587671 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.587672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.587680 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.587721 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:02.587850 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.588508 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.589844 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.590318 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.590379 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.590451 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.590864 (+ 413us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.590905 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:15:02.591027 16247 raft_consensus.cc:2454] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581d3a385f7f49f99ce2e13a3f1ef4bc in term 1.
I20260923 22:15:02.591269 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "482d0242ef0e43efa20df2b1131db29a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" is_pre_election: true
I20260923 22:15:02.591292 15835 catalog_manager.cc:5495] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129). New cstate: current_term: 1 leader_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } health_report { overall_health: HEALTHY } } }
I20260923 22:15:02.591557 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e79c6a487b0461fbf695463124f3366" candidate_uuid: "482d0242ef0e43efa20df2b1131db29a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71c72288c5e743e999537b38cf3a9e4d" is_pre_election: true
I20260923 22:15:02.591637 16247 raft_consensus.cc:2379] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482d0242ef0e43efa20df2b1131db29a in current term 1: Already voted for candidate 581d3a385f7f49f99ce2e13a3f1ef4bc in this term.
I20260923 22:15:02.591866 16048 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482d0242ef0e43efa20df2b1131db29a; no voters: 581d3a385f7f49f99ce2e13a3f1ef4bc, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:02.591991 16308 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 22:15:02.598924 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.596717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.1.132.129:42073)
0923 22:15:02.596774 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:02.596776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:02.596784 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:02.596821 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:02.596822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:02.596841 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:02.596902 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:02.597011 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.597506 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.598205 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:02.598421 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:02.598474 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:02.598624 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:02.598729 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:02.598789 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:02.598829 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 22:15:02.598847 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:15:02.600307 16247 raft_consensus.cc:1261] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Refusing update from remote peer 581d3a385f7f49f99ce2e13a3f1ef4bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:02.600348 16115 raft_consensus.cc:1261] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Refusing update from remote peer 581d3a385f7f49f99ce2e13a3f1ef4bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:02.600603 16312 consensus_queue.cc:1035] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:02.600720 16029 consensus_queue.cc:1035] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:02.605507 15835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:02.605593 15835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:02.606251 15835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Sending DeleteTablet for 3 replicas of tablet 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:02.606531 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e79c6a487b0461fbf695463124f3366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:02 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:33840
I20260923 22:15:02.606580 16095 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e79c6a487b0461fbf695463124f3366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:02 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:46930
I20260923 22:15:02.606595 16227 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e79c6a487b0461fbf695463124f3366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:02 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:40848
I20260923 22:15:02.606799 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15873
I20260923 22:15:02.607386 16326 tablet_replica.cc:330] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: stopping tablet replica
I20260923 22:15:02.607489 16326 raft_consensus.cc:2229] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:02.607594 16326 raft_consensus.cc:2258] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:02.607753 16327 tablet_replica.cc:330] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: stopping tablet replica
I20260923 22:15:02.607846 16327 raft_consensus.cc:2229] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:02.607935 16327 raft_consensus.cc:2258] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:02.608024 16326 ts_tablet_manager.cc:1905] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:02.608300 16327 ts_tablet_manager.cc:1905] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:02.609876 16327 ts_tablet_manager.cc:1918] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:02.609905 16326 ts_tablet_manager.cc:1918] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:02.609944 16327 log.cc:1196] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/9e79c6a487b0461fbf695463124f3366
I20260923 22:15:02.609983 16326 log.cc:1196] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/9e79c6a487b0461fbf695463124f3366
I20260923 22:15:02.610211 16327 ts_tablet_manager.cc:1939] T 9e79c6a487b0461fbf695463124f3366 P 482d0242ef0e43efa20df2b1131db29a: Deleting consensus metadata
I20260923 22:15:02.610240 16326 ts_tablet_manager.cc:1939] T 9e79c6a487b0461fbf695463124f3366 P 71c72288c5e743e999537b38cf3a9e4d: Deleting consensus metadata
I20260923 22:15:02.610605 15819 catalog_manager.cc:4841] TS 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): tablet 9e79c6a487b0461fbf695463124f3366 (table test-table [id=fbb8e5419ed849e79c361d78cb8d9e69]) successfully deleted
I20260923 22:15:02.610687 15819 catalog_manager.cc:4841] TS 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): tablet 9e79c6a487b0461fbf695463124f3366 (table test-table [id=fbb8e5419ed849e79c361d78cb8d9e69]) successfully deleted
W20260923 22:15:02.612601 15820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:42073 (error 108)
I20260923 22:15:02.613030 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.612912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:42073 (local address 127.0.0.1:33854)
0923 22:15:02.612960 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.612997 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:42073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:15:02.613155 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16028
W20260923 22:15:02.613132 15820 catalog_manager.cc:4584] TS 581d3a385f7f49f99ce2e13a3f1ef4bc (127.1.132.129:42073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e79c6a487b0461fbf695463124f3366: Network error: Client connection negotiation failed: client connection to 127.1.132.129:42073: connect: Connection refused (error 111)
I20260923 22:15:02.618125 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16164
I20260923 22:15:02.622062 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 15802
I20260923 22:15:02.626662 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--webserver_interface=127.1.132.190
--webserver_port=45175
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:02.708328 16329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:02.708534 16329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:02.708565 16329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:02.709714 16329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:02.709761 16329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:02.709775 16329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:02.709786 16329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:02.709796 16329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:02.711274 16329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:02.711509 16329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:02.711731 16329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:02.716522 16335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.716693 16334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.716692 16337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.716910 16329 server_base.cc:1034] running on GCE node
I20260923 22:15:02.717141 16329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:02.717470 16329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:02.718655 16329 hybrid_clock.cc:648] HybridClock initialized: now 1790201702718635 us; error 28 us; skew 500 ppm
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:02.721422 16329 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:15:02.721925 16329 webserver.cc:460] Webserver started at http://127.1.132.190:45175/ using document root <none> and password file <none>
I20260923 22:15:02.722123 16329 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:02.722163 16329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:02.723623 16329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:02.724376 16344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.724659 16329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:02.724749 16329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "16e431d93dae4c85acb5e4569b5bcdef"
format_stamp: "Formatted at 2026-09-23 22:15:00 on dist-test-slave-l4q5"
server_key: "1b8a2862ab34ad5b1204c70ed4b064c1"
server_key_iv: "629c2ff4a67671d26201fdd07c49e1fb"
server_key_version: "encryptionkey@0"
I20260923 22:15:02.725142 16329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:02.733527 16329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:02.735461 16329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:02.735585 16329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:02.739342 16329 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:42437
I20260923 22:15:02.739372 16396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:42437 every 8 connection(s)
I20260923 22:15:02.739713 16329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:02.740983 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16329
I20260923 22:15:02.741273 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:42073
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=34851
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:02.741405 16397 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:15:02.742198 16397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap starting.
I20260923 22:15:02.750134 16397 log.cc:824] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:02.752417 16397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:02.752717 16397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef: Bootstrap complete.
I20260923 22:15:02.754778 16397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:02.754993 16397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.755129 16397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:02.755188 16397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:02.755229 16397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:02.755272 16397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:15:02.756254 16397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:02.756340 16397 leader_election.cc:302] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e431d93dae4c85acb5e4569b5bcdef; no voters:
I20260923 22:15:02.756489 16397 leader_election.cc:288] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:15:02.756537 16401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:15:02.756649 16401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [term 3 LEADER]: Becoming Leader. State: Replica: 16e431d93dae4c85acb5e4569b5bcdef, State: Running, Role: LEADER
I20260923 22:15:02.756753 16397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:02.756739 16401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } }
I20260923 22:15:02.757045 16401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e431d93dae4c85acb5e4569b5bcdef. Latest consensus state: current_term: 3 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:02.757131 16401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:02.757333 16403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "16e431d93dae4c85acb5e4569b5bcdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e431d93dae4c85acb5e4569b5bcdef" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42437 } } }
I20260923 22:15:02.757445 16403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e431d93dae4c85acb5e4569b5bcdef [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:02.757580 16406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:02.758378 16406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:02.758492 16406 catalog_manager.cc:1259] Loaded cluster ID: 92df15bd11fa47bd9cff516b2260bf4c
I20260923 22:15:02.758569 16406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 22:15:02.758997 16406 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6f1be469ad google::LogMessage::Fail() at ??:0
@ 0x7f6f1be4988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6f1be464de google::LogMessage::Flush() at ??:0
@ 0x7f6f1be47a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6f23d6b0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6f23d748b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6f1dd140d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6f1dd09954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6f1eb6d6db start_thread at ??:0
@ 0x7f6f1a94461f clone at ??:0
W20260923 22:15:02.855839 16399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:02.856160 16399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:02.856235 16399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:02.858100 16399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:02.858180 16399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:02.858222 16399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:15:02.860656 16399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:42073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=34851
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:02.861142 16399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:02.861511 16399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:02.867071 16433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:02.867097 16436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.867317 16399 server_base.cc:1034] running on GCE node
W20260923 22:15:02.867071 16434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:02.867576 16399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:02.867842 16399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:02.869016 16399 hybrid_clock.cc:648] HybridClock initialized: now 1790201702868997 us; error 33 us; skew 500 ppm
Sep 23 22:15:02 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:02.871733 16399 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:15:02.872267 16399 webserver.cc:460] Webserver started at http://127.1.132.129:34851/ using document root <none> and password file <none>
I20260923 22:15:02.872538 16399 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:02.872604 16399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:02.874524 16399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:15:02.875355 16443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:02.875561 16399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:02.875656 16399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "7da7e434cd51b76f9ccc74875befb333"
server_key_iv: "d3046e74e942258a5caaa2c368ac9aaa"
server_key_version: "encryptionkey@0"
I20260923 22:15:02.876029 16399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:02.903836 16399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:02.906032 16399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:02.906180 16399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:02.906417 16399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:02.906886 16450 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:15:02.908937 16399 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:15:02.909003 16399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:15:02.909041 16399 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:15:02.909741 16399 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:15:02.909777 16399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:02.909875 16450 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap starting.
I20260923 22:15:02.916508 16399 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:42073
I20260923 22:15:02.916646 16557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:42073 every 8 connection(s)
I20260923 22:15:02.916966 16399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:02.920792 16450 log.cc:824] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:02.923777 16450 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:02.924104 16450 tablet_bootstrap.cc:492] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Bootstrap complete.
I20260923 22:15:02.925259 16450 ts_tablet_manager.cc:1397] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.003s
I20260923 22:15:02.926994 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16399
I20260923 22:15:02.926779 16450 raft_consensus.cc:348] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.927136 16450 raft_consensus.cc:729] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581d3a385f7f49f99ce2e13a3f1ef4bc, State: Initialized, Role: FOLLOWER
I20260923 22:15:02.927250 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:42899
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=42115
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:02.927318 16450 consensus_queue.cc:260] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:02.927623 16450 ts_tablet_manager.cc:1428] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260923 22:15:02.983924 16560 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:15:02.917614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:39915)
0923 22:15:02.917781 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.917827 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:02.918287 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:02.983697 (+ 65410us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260923 22:15:02.984230 16558 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:42437 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:42437: Client connection negotiation failed: client connection to 127.1.132.190:42437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:15:02.984587 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.984462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:56617)
0923 22:15:02.984513 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.984561 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:15:02.984963 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:02.984879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:59839)
0923 22:15:02.984918 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:02.984945 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:15:02.985106 16558 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:15:03.011813 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:03.012042 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:03.012079 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:03.013501 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:03.013578 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:03.013622 16563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:15:03.015228 16563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:42899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=42115
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:03.015482 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:03.015702 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:03.021587 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:03.021670 16563 server_base.cc:1034] running on GCE node
W20260923 22:15:03.021627 16569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:03.021636 16568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:03.022083 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:03.022339 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:03.023501 16563 hybrid_clock.cc:648] HybridClock initialized: now 1790201703023488 us; error 32 us; skew 500 ppm
Sep 23 22:15:03 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:03.026183 16563 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:15:03.026692 16563 webserver.cc:460] Webserver started at http://127.1.132.130:42115/ using document root <none> and password file <none>
I20260923 22:15:03.026898 16563 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:03.026948 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:03.028322 16563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:03.028911 16578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.029054 16563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.029145 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "482d0242ef0e43efa20df2b1131db29a"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "016f11fbcdfbcaad871a71c4500c5d75"
server_key_iv: "894d2c028a636d019e81135a82ef1c27"
server_key_version: "encryptionkey@0"
I20260923 22:15:03.029484 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:03.044042 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:03.046255 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:03.046401 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:03.046633 16563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:03.046926 16563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:03.046958 16563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.046988 16563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:03.047004 16563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.052652 16563 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:42899
I20260923 22:15:03.052732 16691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:42899 every 8 connection(s)
I20260923 22:15:03.053051 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:15:03.054140 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.053746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:37795)
0923 22:15:03.053906 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.053985 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260923 22:15:03.054595 16692 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:42437 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:42437: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
I20260923 22:15:03.054920 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.054801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:43107)
0923 22:15:03.054864 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.054896 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:15:03.055258 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.055185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:41467)
0923 22:15:03.055220 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.055242 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 22:15:03.055368 16692 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:15:03.062553 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16563
I20260923 22:15:03.062783 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:38745
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:42437
--webserver_port=36799
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:03.146522 16695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:03.146750 16695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:03.146785 16695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:03.147977 16695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:03.148022 16695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:03.148058 16695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:15:03.149567 16695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:38745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=36799
--tserver_master_addrs=127.1.132.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:03.149812 16695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:03.150024 16695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:03.155437 16701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:03.155525 16700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:03.155403 16703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:03.155647 16695 server_base.cc:1034] running on GCE node
I20260923 22:15:03.155839 16695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:03.156075 16695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:03.157279 16695 hybrid_clock.cc:648] HybridClock initialized: now 1790201703157262 us; error 31 us; skew 500 ppm
Sep 23 22:15:03 dist-test-slave-l4q5 krb5kdc[15257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201703, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:03.159767 16695 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:15:03.160383 16695 webserver.cc:460] Webserver started at http://127.1.132.131:36799/ using document root <none> and password file <none>
I20260923 22:15:03.160624 16695 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:03.160672 16695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:03.162060 16695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:03.162719 16710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.162904 16695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:03.163005 16695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "71c72288c5e743e999537b38cf3a9e4d"
format_stamp: "Formatted at 2026-09-23 22:15:01 on dist-test-slave-l4q5"
server_key: "f78d37742639cc2ea00a775e58eba5d7"
server_key_iv: "82616ef0743374ca090fc336eee7bd56"
server_key_version: "encryptionkey@0"
I20260923 22:15:03.163353 16695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:03.196688 16695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:03.199163 16695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:03.199288 16695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:03.199484 16695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:03.199795 16695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:03.199828 16695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.199858 16695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:03.199883 16695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:03.206432 16695 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:38745
I20260923 22:15:03.206492 16823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:38745 every 8 connection(s)
I20260923 22:15:03.206852 16695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:15:03.208214 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.207719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:59755)
0923 22:15:03.207918 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.208017 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260923 22:15:03.208284 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16695
W20260923 22:15:03.208648 16824 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:42437 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:42437: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
I20260923 22:15:03.208979 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.208855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:48861)
0923 22:15:03.208903 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.208937 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:15:03.209260 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.209166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:55145)
0923 22:15:03.209197 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.209240 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260923 22:15:03.209344 16824 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:15:03.985744 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:03.985480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:33131)
0923 22:15:03.985646 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:03.985682 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260923 22:15:04.055984 16829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.055744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:41945)
0923 22:15:04.055883 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.055927 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260923 22:15:04.209980 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.209723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:55205)
0923 22:15:04.209855 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.209904 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
I20260923 22:15:04.261315 16831 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:04.261440 16831 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:04.261770 16831 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
Sep 23 22:15:04 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.131@KRBTEST.COM
Sep 23 22:15:04 dist-test-slave-l4q5 krb5kdc[15257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201702, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.130@KRBTEST.COM
I20260923 22:15:04.272496 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.262199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:39575 (local address 127.1.132.131:38745)
0923 22:15:04.262328 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:04.262336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:04.262365 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:15:04.262936 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:04.262939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:04.262999 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:04.263151 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:04.263987 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.264894 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.265984 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.266376 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.266476 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:04.268785 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:04.268790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:04.268791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:04.268805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:04.270653 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271308 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:04.271310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:04.271311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:04.271387 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271557 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:04.271558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:04.271559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:04.271815 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:04.271884 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:04.272245 (+ 361us) server_negotiation.cc:299] Negotiation successful
0923 22:15:04.272380 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260923 22:15:04.272496 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.262110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:38445 (local address 127.1.132.130:42899)
0923 22:15:04.262266 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:04.262278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:04.262322 (+ 44us) server_negotiation.cc:407] Connection header received
0923 22:15:04.262936 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:04.262939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:04.262998 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:04.263151 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:04.263998 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.264982 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.265982 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.266406 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.266500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:04.269128 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:04.269133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:04.269133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:04.269146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:04.270768 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271156 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:04.271158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:04.271158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:04.271238 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271467 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:04.271468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:04.271468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:04.271773 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:04.271863 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:04.272245 (+ 382us) server_negotiation.cc:299] Negotiation successful
0923 22:15:04.272379 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260923 22:15:04.272508 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.262110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:38745 (local address 127.1.132.129:39575)
0923 22:15:04.262234 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.262243 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:04.262294 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:04.263287 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:04.263288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:04.263672 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:04.263863 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:04.263869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.264996 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.264997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:04.265905 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:04.265908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.266454 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.266455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:04.266565 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:04.267151 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:04.267157 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:04.268664 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:04.270737 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:04.270745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:04.270747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:04.271092 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:04.271444 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:04.271445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:04.271501 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:04.271882 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:04.271882 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:04.272042 (+ 160us) client_negotiation.cc:769] Sending connection context
0923 22:15:04.272159 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 22:15:04.272402 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":76,"thread_start_us":73,"threads_started":1}
I20260923 22:15:04.272902 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.261974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:42899 (local address 127.1.132.129:38445)
0923 22:15:04.262050 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.262066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:04.262141 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:04.263208 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:04.263211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:04.263671 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:04.263863 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:04.263869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.265063 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.265064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:04.265906 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:04.265909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:04.266471 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:04.266472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:04.266566 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:04.267139 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:04.267177 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:04.269013 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:04.270848 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:04.270848 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:04.270849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:04.271092 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:04.271292 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:04.271292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:04.271293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:04.271396 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:04.271868 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:04.271876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:04.272042 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 22:15:04.272159 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 22:15:04.272471 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":173}
W20260923 22:15:04.273357 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
W20260923 22:15:04.273613 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:04.273659 16444 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters: 482d0242ef0e43efa20df2b1131db29a, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:04.273770 16831 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:04.986464 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:04.986244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:52585)
0923 22:15:04.986375 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:04.986415 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260923 22:15:05.056771 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.056511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:49827)
0923 22:15:05.056678 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.056725 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":61,"threads_started":1}
I20260923 22:15:05.210615 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.210393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:51081)
0923 22:15:05.210538 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.210573 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260923 22:15:05.987205 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:05.986952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:46953)
0923 22:15:05.987103 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:05.987158 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260923 22:15:06.030709 16839 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:06.030803 16839 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:06.030949 16839 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
W20260923 22:15:06.031343 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
W20260923 22:15:06.031430 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:06.031450 16444 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters: 482d0242ef0e43efa20df2b1131db29a, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:06.031549 16839 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:06.057591 16840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.057326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:44197)
0923 22:15:06.057499 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.057545 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260923 22:15:06.211261 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.211074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:42959)
0923 22:15:06.211189 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.211218 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260923 22:15:06.987968 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:06.987703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:51841)
0923 22:15:06.987884 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:06.987922 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260923 22:15:07.058511 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.058221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:59801)
0923 22:15:07.058419 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.058463 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
I20260923 22:15:07.214488 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.214213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:42067)
0923 22:15:07.214369 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.214429 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260923 22:15:07.988801 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:07.988553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:52193)
0923 22:15:07.988699 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:07.988754 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260923 22:15:08.059350 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.059068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:43325)
0923 22:15:08.059222 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.059268 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260923 22:15:08.215248 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.214999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:43341)
0923 22:15:08.215154 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.215195 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260923 22:15:08.452140 16848 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:08.452284 16848 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:08.452420 16848 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
W20260923 22:15:08.452823 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
W20260923 22:15:08.452894 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:08.452920 16444 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters: 482d0242ef0e43efa20df2b1131db29a, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:08.453006 16848 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:08.989619 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:08.989390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:49541)
0923 22:15:08.989529 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:08.989571 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260923 22:15:09.060322 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:09.060004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:43629)
0923 22:15:09.060176 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:09.060263 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260923 22:15:09.215965 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:09.215734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:44581)
0923 22:15:09.215878 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:09.215920 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260923 22:15:09.990311 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:09.990107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:39943)
0923 22:15:09.990233 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:09.990266 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260923 22:15:10.061095 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.060844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:44907)
0923 22:15:10.060998 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.061048 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260923 22:15:10.216741 16854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.216532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:40165)
0923 22:15:10.216655 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.216696 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260923 22:15:10.991024 16855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:10.990787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:36837)
0923 22:15:10.990923 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:10.990972 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260923 22:15:11.061903 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:11.061640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:52785)
0923 22:15:11.061809 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:11.061856 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
I20260923 22:15:11.217685 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:11.217344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:58691)
0923 22:15:11.217522 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:11.217561 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260923 22:15:11.991808 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:11.991575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:47601)
0923 22:15:11.991712 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:11.991759 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260923 22:15:12.062664 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:12.062404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:34493)
0923 22:15:12.062570 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:12.062616 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260923 22:15:12.218513 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:12.218282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:40895)
0923 22:15:12.218433 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:12.218467 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260923 22:15:12.766876 16861 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:12.766968 16861 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:12.767117 16861 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
W20260923 22:15:12.767514 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
W20260923 22:15:12.767575 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:12.767596 16444 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters: 482d0242ef0e43efa20df2b1131db29a, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:12.767665 16861 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:12.992712 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:12.992464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:42127)
0923 22:15:12.992611 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:12.992660 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260923 22:15:13.063422 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:13.063160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:52071)
0923 22:15:13.063305 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:13.063357 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260923 22:15:13.219308 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:13.219034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:32971)
0923 22:15:13.219191 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:13.219244 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260923 22:15:13.993589 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:13.993267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:57287)
0923 22:15:13.993487 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:13.993538 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260923 22:15:14.064358 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:14.063987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:59743)
0923 22:15:14.064176 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:14.064306 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260923 22:15:14.220067 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:14.219829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:60279)
0923 22:15:14.219978 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:14.220019 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260923 22:15:14.829411 16868 raft_consensus.cc:482] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:14.829509 16868 raft_consensus.cc:504] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482d0242ef0e43efa20df2b1131db29a" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 42899 } } peers { permanent_uuid: "71c72288c5e743e999537b38cf3a9e4d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 38745 } } peers { permanent_uuid: "581d3a385f7f49f99ce2e13a3f1ef4bc" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 42073 } }
I20260923 22:15:14.829658 16868 leader_election.cc:288] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899), 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745)
W20260923 22:15:14.830075 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71c72288c5e743e999537b38cf3a9e4d (127.1.132.131:38745): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
W20260923 22:15:14.830145 16444 leader_election.cc:341] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482d0242ef0e43efa20df2b1131db29a (127.1.132.130:42899): Not found: Tablet not found: 9e79c6a487b0461fbf695463124f3366
I20260923 22:15:14.830165 16444 leader_election.cc:302] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 581d3a385f7f49f99ce2e13a3f1ef4bc; no voters: 482d0242ef0e43efa20df2b1131db29a, 71c72288c5e743e999537b38cf3a9e4d
I20260923 22:15:14.830271 16868 raft_consensus.cc:2726] T 9e79c6a487b0461fbf695463124f3366 P 581d3a385f7f49f99ce2e13a3f1ef4bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:15:14.994418 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:14.994121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:58131)
0923 22:15:14.994304 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:14.994370 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260923 22:15:15.065323 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:15.064994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:35213)
0923 22:15:15.065167 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:15.065257 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260923 22:15:15.220813 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:15.220593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:36625)
0923 22:15:15.220713 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:15.220766 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260923 22:15:15.995262 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:15.994981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:45977)
0923 22:15:15.995156 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:15.995210 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260923 22:15:16.066148 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:16.065859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:42355)
0923 22:15:16.066050 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:16.066100 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
I20260923 22:15:16.221621 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:16.221344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:59403)
0923 22:15:16.221505 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:16.221564 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260923 22:15:16.996026 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:16.995770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:45159)
0923 22:15:16.995933 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:16.995978 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260923 22:15:17.066943 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:17.066724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:51049)
0923 22:15:17.066847 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:17.066882 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260923 22:15:17.222411 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:17.222137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:41139)
0923 22:15:17.222313 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:17.222362 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260923 22:15:17.996906 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:17.996639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.129:53939)
0923 22:15:17.996819 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:17.996858 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260923 22:15:18.067715 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:18.067497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.130:53375)
0923 22:15:18.067626 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:18.067670 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260923 22:15:18.215150 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16399
I20260923 22:15:18.219507 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16563
I20260923 22:15:18.223213 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:18.223014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42437 (local address 127.1.132.131:38343)
0923 22:15:18.223133 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:18.223170 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260923 22:15:18.223546 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16695
2026-09-23T22:15:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18407 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:15:18 dist-test-slave-l4q5 krb5kdc[16883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:15:18 dist-test-slave-l4q5 krb5kdc[16883](info): set up 2 sockets
Sep 23 22:15:18 dist-test-slave-l4q5 krb5kdc[16883](info): commencing operation
krb5kdc: starting...
W20260923 22:15:19.180006 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:15:19.305208 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:19.386224 16896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:19.386458 16896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:19.386547 16896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:19.387827 16896 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:15:19.387879 16896 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:15:19.387892 16896 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:15:19.387902 16896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:19.387913 16896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:19.387925 16896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:19.387935 16896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:19.387946 16896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:19.389539 16896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:19.389786 16896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:19.390004 16896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:19.395015 16902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.395002 16901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.395064 16896 server_base.cc:1034] running on GCE node
W20260923 22:15:19.395006 16904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.395515 16896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:19.395788 16896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:19.396939 16896 hybrid_clock.cc:648] HybridClock initialized: now 1790201719396925 us; error 31 us; skew 500 ppm
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:19.399320 16896 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:15:19.399708 16896 webserver.cc:460] Webserver started at http://127.1.132.190:40795/ using document root <none> and password file <none>
I20260923 22:15:19.399879 16896 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:19.399919 16896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:19.400034 16896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:19.401124 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "565f1b0c876941c3ae906d180a0d67fa"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "101b4394f10c439f9f2161844c9d8803"
server_key_iv: "a47afa84a41d4d25f6d51bd18940138d"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.401448 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "565f1b0c876941c3ae906d180a0d67fa"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "101b4394f10c439f9f2161844c9d8803"
server_key_iv: "a47afa84a41d4d25f6d51bd18940138d"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.402767 16896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:15:19.403486 16911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.403635 16896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:19.403743 16896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "565f1b0c876941c3ae906d180a0d67fa"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "101b4394f10c439f9f2161844c9d8803"
server_key_iv: "a47afa84a41d4d25f6d51bd18940138d"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.403805 16896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:19.413496 16896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:19.415540 16896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:19.415683 16896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:19.419334 16896 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40719
I20260923 22:15:19.419373 16963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40719 every 8 connection(s)
I20260923 22:15:19.419689 16896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:19.420339 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16896
I20260923 22:15:19.420432 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:15:19.420627 1554 external_mini_cluster.cc:1428] Setting key 3a3169bedb2669b5b50b4bae66b7a229
I20260923 22:15:19.420660 16964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:19.423761 16964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap starting.
I20260923 22:15:19.424522 16964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:19.424820 16964 log.cc:824] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:19.425506 16964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: No bootstrap required, opened a new log
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:19.427451 16964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:19.427640 16964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:19.427699 16964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Initialized, Role: FOLLOWER
I20260923 22:15:19.427821 16964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:19.427896 16964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:19.427935 16964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:19.427979 16964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:19.428561 16964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:19.428658 16964 leader_election.cc:302] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 565f1b0c876941c3ae906d180a0d67fa; no voters:
I20260923 22:15:19.428817 16964 leader_election.cc:288] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:15:19.428869 16969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:19.428997 16969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 LEADER]: Becoming Leader. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Running, Role: LEADER
I20260923 22:15:19.429071 16964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:19.429119 16969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:19.429473 16969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565f1b0c876941c3ae906d180a0d67fa. Latest consensus state: current_term: 1 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:15:19.429482 16970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:15:19.429636 16970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:19.429630 16969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:19.429930 16978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:19.430423 16978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:19.431501 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.421144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.1.132.190:40719)
0923 22:15:19.421304 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.421311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.421335 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:15:19.421712 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.421719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.421765 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.421873 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:19.422617 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.423560 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.424656 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.425099 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.425208 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.427404 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:19.427408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:19.427409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:19.427421 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:19.429346 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.429782 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.429786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.429786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.429897 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.430112 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.430113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.430114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.430452 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:19.430607 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.431183 (+ 576us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.431315 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260923 22:15:19.433161 16978 catalog_manager.cc:1347] Generated new cluster ID: 37ab16b30eaa4237ac9dc3f4f26706d8
I20260923 22:15:19.433205 16978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:19.460251 16978 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:15:19.460800 16978 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:19.469080 16978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Generated new TSK 0
I20260923 22:15:19.469405 16978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:15:19.506522 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:19.589115 16990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:19.589355 16990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:19.589382 16990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:19.590584 16990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:19.590631 16990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:19.590672 16990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:15:19.592263 16990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:19.592480 16990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:19.592664 16990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:19.597556 16995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.597561 16996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.597549 16998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.597915 16990 server_base.cc:1034] running on GCE node
I20260923 22:15:19.598127 16990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:19.598461 16990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:19.599639 16990 hybrid_clock.cc:648] HybridClock initialized: now 1790201719599622 us; error 35 us; skew 500 ppm
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:19.602463 16990 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:15:19.602929 16990 webserver.cc:460] Webserver started at http://127.1.132.129:45969/ using document root <none> and password file <none>
I20260923 22:15:19.603142 16990 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:19.603191 16990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:19.603317 16990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:19.604427 16990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "277c2b8d4f0145d7be03f31e97a53622"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "367b6f2e7645f3fdf3a14ef385d14699"
server_key_iv: "f32b08e1f8afd0229956fcd1cc49f4ac"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.604769 16990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "277c2b8d4f0145d7be03f31e97a53622"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "367b6f2e7645f3fdf3a14ef385d14699"
server_key_iv: "f32b08e1f8afd0229956fcd1cc49f4ac"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.606470 16990 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:19.607368 17005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.607678 16990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:19.607771 16990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "277c2b8d4f0145d7be03f31e97a53622"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "367b6f2e7645f3fdf3a14ef385d14699"
server_key_iv: "f32b08e1f8afd0229956fcd1cc49f4ac"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.607851 16990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:19.626407 16990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:19.629205 16990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:19.629359 16990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:19.629655 16990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:19.629980 16990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:19.630016 16990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.630045 16990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:19.630064 16990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.636106 16990 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:43409
I20260923 22:15:19.636240 17118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:43409 every 8 connection(s)
I20260923 22:15:19.636515 16990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:19.641568 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 16990
I20260923 22:15:19.641686 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:15:19.641896 1554 external_mini_cluster.cc:1428] Setting key 1c5145045c6fd9d7d98b64d9affb6cb3
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:19.646795 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.637254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:57221 (local address 127.1.132.190:40719)
0923 22:15:19.637363 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.637366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.637525 (+ 159us) server_negotiation.cc:407] Connection header received
0923 22:15:19.638067 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.638069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.638081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.638138 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:19.639110 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.639636 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.640679 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.640893 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.640951 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.643526 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:19.643528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:19.643528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:19.643538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:19.645151 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.645620 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.645621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.645622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.645686 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.645990 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.645991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.645991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.646137 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:19.646205 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.646590 (+ 385us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.646692 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 22:15:19.646895 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.637187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:57221)
0923 22:15:19.637396 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.637438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.637961 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.638228 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.638235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.638813 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:19.639012 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.639017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.639725 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.639726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.640599 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.640602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.640965 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.640966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.641087 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.641593 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:19.641603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:19.643348 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:19.645284 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.645292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.645295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.645547 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.645787 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.645788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.645788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.645924 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.646265 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:19.646276 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:19.646450 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.646552 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.646704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260923 22:15:19.647482 17119 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:19.647603 17119 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:19.647825 17119 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:19.648411 16928 ts_manager.cc:194] Registered new tserver with Master: 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409)
I20260923 22:15:19.649051 16928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:57221
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:15:19.671624 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:19.750717 17124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:19.750936 17124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:19.750959 17124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:19.752161 17124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:19.752239 17124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:19.752265 17124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:15:19.753738 17124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:19.753961 17124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:19.754184 17124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:15:19.760603 17124 server_base.cc:1034] running on GCE node
W20260923 22:15:19.760568 17129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.760576 17130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.760552 17132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.761492 17124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:19.761756 17124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:19.762928 17124 hybrid_clock.cc:648] HybridClock initialized: now 1790201719762916 us; error 32 us; skew 500 ppm
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:19.765496 17124 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:15:19.765934 17124 webserver.cc:460] Webserver started at http://127.1.132.130:40769/ using document root <none> and password file <none>
I20260923 22:15:19.766129 17124 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:19.766189 17124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:19.766297 17124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:19.767292 17124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "59faf951892d4202a250465bfc2869e3"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "334e8170159ad0b6d2650f8f53282cb0"
server_key_iv: "5791043ca1a0c0082f24f7ff8855c525"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.767589 17124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "59faf951892d4202a250465bfc2869e3"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "334e8170159ad0b6d2650f8f53282cb0"
server_key_iv: "5791043ca1a0c0082f24f7ff8855c525"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.768853 17124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:15:19.769536 17139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.769708 17124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.769786 17124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "59faf951892d4202a250465bfc2869e3"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "334e8170159ad0b6d2650f8f53282cb0"
server_key_iv: "5791043ca1a0c0082f24f7ff8855c525"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.769850 17124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:19.782104 17124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:19.784379 17124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:19.784528 17124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:19.784767 17124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:19.785066 17124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:19.785099 17124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.785130 17124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:19.785166 17124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.790606 17124 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39171
I20260923 22:15:19.790661 17252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39171 every 8 connection(s)
I20260923 22:15:19.790949 17124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:19.796693 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17124
I20260923 22:15:19.796797 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:15:19.796977 1554 external_mini_cluster.cc:1428] Setting key 1964ab5a3fb0fa9cf84f25a57902069a
I20260923 22:15:19.799880 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.791655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:52373 (local address 127.1.132.190:40719)
0923 22:15:19.791708 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.791710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.791903 (+ 193us) server_negotiation.cc:407] Connection header received
0923 22:15:19.792425 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.792426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.792439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.792489 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:19.793392 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.793887 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.794839 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.795034 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.795088 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.797228 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:19.797229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:19.797230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:19.797238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:19.798490 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.798939 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.798941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.798941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.799001 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.799211 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.799212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.799212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.799368 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:19.799430 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.799720 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.799796 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:15:19.800004 17255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.791641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:52373)
0923 22:15:19.791792 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.791827 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.792334 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.792555 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.792560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.793106 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:19.793300 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.793304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.793966 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.793967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.794757 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.794759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.795099 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.795100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.795213 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.795672 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:19.795680 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:19.797131 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:19.798567 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.798573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.798576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.798859 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.799057 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.799057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.799057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.799161 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.799452 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:19.799458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:19.799579 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.799686 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.799811 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260923 22:15:19.800519 17253 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:19.800619 17253 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:19.800796 17253 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:19.801175 16928 ts_manager.cc:194] Registered new tserver with Master: 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171)
I20260923 22:15:19.801544 16928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:52373
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:15:19.823213 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:19.902979 17258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:19.903180 17258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:19.903203 17258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:19.904428 17258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:19.904475 17258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:19.904502 17258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:15:19.905918 17258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:19.906131 17258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:19.906323 17258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:19.911178 17266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:19.911263 17263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.911198 17258 server_base.cc:1034] running on GCE node
W20260923 22:15:19.911335 17264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:19.911592 17258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:19.911808 17258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:19.912941 17258 hybrid_clock.cc:648] HybridClock initialized: now 1790201719912923 us; error 35 us; skew 500 ppm
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:19.915524 17258 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:15:19.915961 17258 webserver.cc:460] Webserver started at http://127.1.132.131:34117/ using document root <none> and password file <none>
I20260923 22:15:19.916204 17258 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:19.916260 17258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:19.916383 17258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:15:19.917389 17258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "9164d545640440dd848673f2a76a966d"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "18066dff4713b5e211e1034237493109"
server_key_iv: "3543b4e513f39915c0d16f1324f243ad"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.917740 17258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "9164d545640440dd848673f2a76a966d"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "18066dff4713b5e211e1034237493109"
server_key_iv: "3543b4e513f39915c0d16f1324f243ad"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.919078 17258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:15:19.919823 17273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.919973 17258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:15:19.920063 17258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9164d545640440dd848673f2a76a966d"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "18066dff4713b5e211e1034237493109"
server_key_iv: "3543b4e513f39915c0d16f1324f243ad"
server_key_version: "encryptionkey@0"
I20260923 22:15:19.920131 17258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:19.931135 17258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:19.933089 17258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:19.933216 17258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:19.933404 17258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:19.933686 17258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:19.933717 17258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.933743 17258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:19.933768 17258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:19.939056 17258 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45605
I20260923 22:15:19.939128 17386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45605 every 8 connection(s)
I20260923 22:15:19.939383 17258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:15:19 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:19.947757 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.940243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:38351 (local address 127.1.132.190:40719)
0923 22:15:19.940305 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.940308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.940346 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:15:19.940827 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.940829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.940840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.940904 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:19.941762 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.942319 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.943336 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.943559 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.943652 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.945645 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:19.945647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:19.945647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:19.945654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:19.946727 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.947047 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.947047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.947048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.947097 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.947278 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.947278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.947278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.947377 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:19.947423 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.947633 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.947693 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:15:19.947921 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.940062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:38351)
0923 22:15:19.940236 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.940277 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.940733 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.940952 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.940955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.941480 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:19.941672 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.941677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.942408 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.942408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.943252 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.943254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.943662 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.943663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.943754 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.944229 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:19.944238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:19.945560 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:19.946792 (+ 1232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.946797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.946799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.946989 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.947150 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:19.947150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:19.947150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:19.947235 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:19.947421 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:19.947424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:19.947529 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.947603 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.947724 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260923 22:15:19.948020 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17258
I20260923 22:15:19.948101 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:15:19.948333 1554 external_mini_cluster.cc:1428] Setting key 322c47d56d399fc83bcb29681d631b23
I20260923 22:15:19.948488 17387 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:19.948570 17387 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:19.948746 17387 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:19.949075 16928 ts_manager.cc:194] Registered new tserver with Master: 9164d545640440dd848673f2a76a966d (127.1.132.131:45605)
I20260923 22:15:19.949405 16928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:38351
I20260923 22:15:19.960079 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:15:19.968103 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.961442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.1.132.190:40719)
0923 22:15:19.961516 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.961518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.961544 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:15:19.961637 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.961638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.961649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.961696 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:19.962236 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.962733 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.963432 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.963621 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.963676 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.964864 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:19.964866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:19.964866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:19.964874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:19.966331 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.966824 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.966826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.966826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.967016 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:19.967253 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:19.967254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:19.967254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:19.967591 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:19.967778 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.967912 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.968007 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:15:19.970338 16928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:19.971954 16928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:19.978789 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45605 (local address 127.0.0.1:48248)
0923 22:15:19.975086 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.975095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.975138 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.975418 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.975418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.975420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:19.975457 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.975458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976367 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.976368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.977600 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.977602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.978502 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.978503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.978570 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.978583 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.978652 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.978698 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":100,"thread_start_us":63,"threads_started":1}
I20260923 22:15:19.978823 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.1.132.131:45605)
0923 22:15:19.975028 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.975032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.975190 (+ 158us) server_negotiation.cc:407] Connection header received
0923 22:15:19.975245 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.975247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.975271 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.975344 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:19.975507 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976270 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.977695 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.978404 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.978483 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.978545 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.978688 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.978735 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":96,"threads_started":1}
I20260923 22:15:19.978976 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.0.0.1:54090)
0923 22:15:19.974933 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.974944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.975005 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.975311 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.975312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.975314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:19.975363 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.975365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976643 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.976644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.977848 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.977851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.978742 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.978744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.978817 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.978820 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.978872 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.978913 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260923 22:15:19.979207 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54090 (local address 127.1.132.129:43409)
0923 22:15:19.974940 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.974944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.975084 (+ 140us) server_negotiation.cc:407] Connection header received
0923 22:15:19.975136 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.975138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.975154 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.975225 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:19.975426 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976538 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.977940 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.978652 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.978721 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.978774 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.979082 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.979130 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260923 22:15:19.979581 17321 tablet_service.cc:1467] Processing CreateTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e (DEFAULT_TABLE table=test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:19.979777 17053 tablet_service.cc:1467] Processing CreateTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e (DEFAULT_TABLE table=test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:19.979959 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7badcb882d41ccaff7444ec76cfa1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:19.980105 17053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7badcb882d41ccaff7444ec76cfa1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:19.980202 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39171 (local address 127.0.0.1:36522)
0923 22:15:19.974775 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:19.974826 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:19.974901 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:19.975199 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:19.975203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:19.975205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:19.975284 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.975286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976757 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.976758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.977849 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:19.977851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.979928 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.979929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:19.980010 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.980015 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:19.980067 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:15:19.980110 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":27,"thread_start_us":70,"threads_started":1}
I20260923 22:15:19.980441 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:19.974613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.1.132.130:39171)
0923 22:15:19.974759 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:19.974764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:19.974981 (+ 217us) server_negotiation.cc:407] Connection header received
0923 22:15:19.975026 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:19.975029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:19.975046 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:19.975110 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:19.975358 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.976537 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.978812 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:19.979774 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:19.979841 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:19.979892 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:19.980088 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 22:15:19.980136 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260923 22:15:19.981065 17187 tablet_service.cc:1467] Processing CreateTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e (DEFAULT_TABLE table=test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:19.981400 17187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7badcb882d41ccaff7444ec76cfa1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:19.982823 17404 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Bootstrap starting.
I20260923 22:15:19.983625 17405 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap starting.
I20260923 22:15:19.983655 17404 tablet_bootstrap.cc:654] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:19.984283 17404 log.cc:824] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:19.984328 17405 tablet_bootstrap.cc:654] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:19.984766 17406 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Bootstrap starting.
I20260923 22:15:19.984860 17405 log.cc:824] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:19.985169 17404 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: No bootstrap required, opened a new log
I20260923 22:15:19.985224 17404 ts_tablet_manager.cc:1397] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:19.985589 17405 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: No bootstrap required, opened a new log
I20260923 22:15:19.985656 17405 ts_tablet_manager.cc:1397] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:15:19.987175 17404 raft_consensus.cc:348] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.987367 17404 raft_consensus.cc:374] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:19.987399 17404 raft_consensus.cc:729] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9164d545640440dd848673f2a76a966d, State: Initialized, Role: FOLLOWER
I20260923 22:15:19.987516 17404 consensus_queue.cc:260] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.987607 17405 raft_consensus.cc:348] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.987777 17405 raft_consensus.cc:374] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:19.987810 17404 ts_tablet_manager.cc:1428] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:15:19.987820 17405 raft_consensus.cc:729] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277c2b8d4f0145d7be03f31e97a53622, State: Initialized, Role: FOLLOWER
I20260923 22:15:19.987835 17387 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
I20260923 22:15:19.987947 17405 consensus_queue.cc:260] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.988256 17405 ts_tablet_manager.cc:1428] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:15:19.988369 17119 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
I20260923 22:15:19.988672 17406 tablet_bootstrap.cc:654] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:19.989177 17406 log.cc:824] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:19.989843 17406 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: No bootstrap required, opened a new log
I20260923 22:15:19.989897 17406 ts_tablet_manager.cc:1397] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 22:15:19.991205 17406 raft_consensus.cc:348] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.991333 17406 raft_consensus.cc:374] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:19.991353 17406 raft_consensus.cc:729] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59faf951892d4202a250465bfc2869e3, State: Initialized, Role: FOLLOWER
I20260923 22:15:19.991449 17406 consensus_queue.cc:260] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:19.991708 17406 ts_tablet_manager.cc:1428] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:19.991730 17253 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
W20260923 22:15:20.041532 17254 tablet.cc:2369] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:20.129273 17409 raft_consensus.cc:482] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:20.129431 17409 raft_consensus.cc:504] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:20.129851 17409 leader_election.cc:288] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409), 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171)
I20260923 22:15:20.133927 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.130008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.1.132.131:47831)
0923 22:15:20.130053 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:20.130065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:20.130111 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:20.130344 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:20.130345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:20.130347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:20.130420 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:20.130422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.131423 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.131424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:20.132808 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:20.132812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.133614 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.133615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:20.133724 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:20.133726 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:15:20.133783 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:15:20.133828 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":87}
I20260923 22:15:20.133944 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.130161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:47831 (local address 127.1.132.129:43409)
0923 22:15:20.130209 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:20.130211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:20.130221 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:20.130251 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:20.130252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:20.130261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:20.130340 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:20.130497 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.131290 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.132917 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.133496 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.133584 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:20.133596 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:20.133824 (+ 228us) server_negotiation.cc:299] Negotiation successful
0923 22:15:20.133876 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:15:20.133976 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.130145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:44497 (local address 127.1.132.130:39171)
0923 22:15:20.130187 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:20.130189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:20.130362 (+ 173us) server_negotiation.cc:407] Connection header received
0923 22:15:20.130397 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:20.130398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:20.130407 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:20.130450 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:20.130566 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.131301 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.132917 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.133532 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.133593 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:20.133601 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:20.133857 (+ 256us) server_negotiation.cc:299] Negotiation successful
0923 22:15:20.133919 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:15:20.133932 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.130071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39171 (local address 127.1.132.131:44497)
0923 22:15:20.130266 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:20.130273 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:20.130323 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:20.130452 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:20.130453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:20.130454 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:20.130507 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:20.130509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.131423 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.131424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:20.132808 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:20.132812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.133616 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.133617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:20.133716 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:20.133721 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:15:20.133790 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:15:20.133827 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":50,"thread_start_us":144,"threads_started":1}
I20260923 22:15:20.134377 17207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7badcb882d41ccaff7444ec76cfa1e" candidate_uuid: "9164d545640440dd848673f2a76a966d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59faf951892d4202a250465bfc2869e3" is_pre_election: true
I20260923 22:15:20.134380 17073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7badcb882d41ccaff7444ec76cfa1e" candidate_uuid: "9164d545640440dd848673f2a76a966d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277c2b8d4f0145d7be03f31e97a53622" is_pre_election: true
I20260923 22:15:20.134528 17073 raft_consensus.cc:2454] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9164d545640440dd848673f2a76a966d in term 0.
I20260923 22:15:20.134528 17207 raft_consensus.cc:2454] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9164d545640440dd848673f2a76a966d in term 0.
I20260923 22:15:20.134763 17274 leader_election.cc:302] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277c2b8d4f0145d7be03f31e97a53622, 9164d545640440dd848673f2a76a966d; no voters:
I20260923 22:15:20.134930 17409 raft_consensus.cc:2781] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:20.134995 17409 raft_consensus.cc:482] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:20.135020 17409 raft_consensus.cc:3037] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:20.135753 17409 raft_consensus.cc:504] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:20.135955 17409 leader_election.cc:288] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [CANDIDATE]: Term 1 election: Requested vote from peers 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409), 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171)
I20260923 22:15:20.136026 17207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7badcb882d41ccaff7444ec76cfa1e" candidate_uuid: "9164d545640440dd848673f2a76a966d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59faf951892d4202a250465bfc2869e3"
I20260923 22:15:20.136077 17073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7badcb882d41ccaff7444ec76cfa1e" candidate_uuid: "9164d545640440dd848673f2a76a966d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277c2b8d4f0145d7be03f31e97a53622"
I20260923 22:15:20.136099 17207 raft_consensus.cc:3037] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:20.136145 17073 raft_consensus.cc:3037] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:20.137023 17207 raft_consensus.cc:2454] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9164d545640440dd848673f2a76a966d in term 1.
I20260923 22:15:20.137017 17073 raft_consensus.cc:2454] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9164d545640440dd848673f2a76a966d in term 1.
W20260923 22:15:20.137218 17120 tablet.cc:2369] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:20.137332 17277 leader_election.cc:302] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59faf951892d4202a250465bfc2869e3, 9164d545640440dd848673f2a76a966d; no voters:
I20260923 22:15:20.137429 17409 raft_consensus.cc:2781] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:20.137584 17409 raft_consensus.cc:686] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 1 LEADER]: Becoming Leader. State: Replica: 9164d545640440dd848673f2a76a966d, State: Running, Role: LEADER
I20260923 22:15:20.137696 17409 consensus_queue.cc:237] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:20.138988 16928 catalog_manager.cc:5495] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9164d545640440dd848673f2a76a966d (127.1.132.131). New cstate: current_term: 1 leader_uuid: "9164d545640440dd848673f2a76a966d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } health_report { overall_health: UNKNOWN } } }
I20260923 22:15:20.164930 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.162518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.1.132.131:45605)
0923 22:15:20.162579 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:20.162581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:20.162589 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:20.162629 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:20.162630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:20.162650 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:20.162751 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:20.162883 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.163374 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.164107 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:20.164391 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:20.164449 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:20.164594 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:20.164718 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:20.164790 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:20.164829 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 22:15:20.164851 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:15:20.167002 17073 raft_consensus.cc:1261] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Refusing update from remote peer 9164d545640440dd848673f2a76a966d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:20.167002 17207 raft_consensus.cc:1261] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Refusing update from remote peer 9164d545640440dd848673f2a76a966d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:20.167347 17409 consensus_queue.cc:1035] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [LEADER]: Connected to new peer: Peer: permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:20.167544 17414 consensus_queue.cc:1035] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [LEADER]: Connected to new peer: Peer: permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:20.169625 17417 mvcc.cc:205] Tried to move back new op lower bound from 7332666245800841216 to 7332666245684809728. Current Snapshot: MvccSnapshot[applied={T|T < 7332666245800841216}]
I20260923 22:15:20.170346 17419 mvcc.cc:205] Tried to move back new op lower bound from 7332666245800841216 to 7332666245684809728. Current Snapshot: MvccSnapshot[applied={T|T < 7332666245800841216}]
I20260923 22:15:20.172063 16927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:20.172151 16927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:20.172933 16927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Sending DeleteTablet for 3 replicas of tablet cb7badcb882d41ccaff7444ec76cfa1e
I20260923 22:15:20.173230 17321 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:20 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:48248
I20260923 22:15:20.173277 17053 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:20 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:54090
I20260923 22:15:20.173352 17187 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:20 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:36522
I20260923 22:15:20.173439 17427 tablet_replica.cc:330] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: stopping tablet replica
I20260923 22:15:20.173524 17427 raft_consensus.cc:2229] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:15:20.173617 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16990
I20260923 22:15:20.173655 17427 raft_consensus.cc:2258] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:20.174796 17427 ts_tablet_manager.cc:1905] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:20.176314 17429 tablet_replica.cc:330] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: stopping tablet replica
I20260923 22:15:20.176435 17429 raft_consensus.cc:2229] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:20.176575 17429 raft_consensus.cc:2258] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:20.176887 17427 ts_tablet_manager.cc:1918] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:20.176990 17427 log.cc:1196] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/cb7badcb882d41ccaff7444ec76cfa1e
I20260923 22:15:20.176990 17429 ts_tablet_manager.cc:1905] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:20.177327 17427 ts_tablet_manager.cc:1939] T cb7badcb882d41ccaff7444ec76cfa1e P 9164d545640440dd848673f2a76a966d: Deleting consensus metadata
I20260923 22:15:20.177762 16913 catalog_manager.cc:4841] TS 9164d545640440dd848673f2a76a966d (127.1.132.131:45605): tablet cb7badcb882d41ccaff7444ec76cfa1e (table test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]) successfully deleted
I20260923 22:15:20.178572 17429 ts_tablet_manager.cc:1918] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:20.178646 17429 log.cc:1196] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/cb7badcb882d41ccaff7444ec76cfa1e
I20260923 22:15:20.178892 17429 ts_tablet_manager.cc:1939] T cb7badcb882d41ccaff7444ec76cfa1e P 59faf951892d4202a250465bfc2869e3: Deleting consensus metadata
I20260923 22:15:20.179379 16915 catalog_manager.cc:4841] TS 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171): tablet cb7badcb882d41ccaff7444ec76cfa1e (table test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]) successfully deleted
W20260923 22:15:20.179427 16912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:43409 (error 108)
I20260923 22:15:20.179663 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17124
I20260923 22:15:20.180455 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:20.179774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.0.0.1:54092)
0923 22:15:20.180386 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:20.180429 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589}
W20260923 22:15:20.180796 16912 catalog_manager.cc:4584] TS 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb7badcb882d41ccaff7444ec76cfa1e: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43409: connect: Connection refused (error 111)
I20260923 22:15:20.185793 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17258
I20260923 22:15:20.190444 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 16896
I20260923 22:15:40.195178 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--webserver_interface=127.1.132.190
--webserver_port=40795
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:15:40.278329 17431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:40.278554 17431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:40.278576 17431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:40.279755 17431 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:15:40.279794 17431 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:15:40.279806 17431 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:15:40.279817 17431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:15:40.279829 17431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:40.279840 17431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:40.279852 17431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:15:40.279877 17431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:15:40.281390 17431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=40795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:40.281636 17431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:40.281854 17431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:40.290544 17437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.290602 17438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.290701 17440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:40.290726 17431 server_base.cc:1034] running on GCE node
I20260923 22:15:40.291184 17431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:40.291507 17431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:40.292685 17431 hybrid_clock.cc:648] HybridClock initialized: now 1790201740292661 us; error 36 us; skew 500 ppm
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:40.299441 17431 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:15:40.299947 17431 webserver.cc:460] Webserver started at http://127.1.132.190:40795/ using document root <none> and password file <none>
I20260923 22:15:40.300163 17431 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:40.300258 17431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:40.301690 17431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.302424 17447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.302608 17431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.302695 17431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "565f1b0c876941c3ae906d180a0d67fa"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "101b4394f10c439f9f2161844c9d8803"
server_key_iv: "a47afa84a41d4d25f6d51bd18940138d"
server_key_version: "encryptionkey@0"
I20260923 22:15:40.303121 17431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:40.311054 17431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:40.313222 17431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:40.313395 17431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:40.317255 17431 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40719
I20260923 22:15:40.317286 17499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40719 every 8 connection(s)
I20260923 22:15:40.317667 17431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:15:40.319171 17500 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:15:40.319774 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap starting.
I20260923 22:15:40.320266 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17431
I20260923 22:15:40.320499 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:43409
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=45969
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:15:40.328541 17500 log.cc:824] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:40.330399 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:40.330689 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap complete.
I20260923 22:15:40.332867 17500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:40.333112 17500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Initialized, Role: FOLLOWER
I20260923 22:15:40.333271 17500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:40.333329 17500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:15:40.333372 17500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:15:40.333419 17500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:15:40.334415 17500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:40.334506 17500 leader_election.cc:302] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 565f1b0c876941c3ae906d180a0d67fa; no voters:
I20260923 22:15:40.334661 17500 leader_election.cc:288] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:15:40.334710 17504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:15:40.334834 17504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 LEADER]: Becoming Leader. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Running, Role: LEADER
I20260923 22:15:40.334939 17500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:15:40.334983 17504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:15:40.335330 17504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565f1b0c876941c3ae906d180a0d67fa. Latest consensus state: current_term: 2 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:15:40.335405 17504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:40.335564 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:15:40.335640 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:15:40.335863 17512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:15:40.336771 17512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:15:40.336915 17512 catalog_manager.cc:1259] Loaded cluster ID: 37ab16b30eaa4237ac9dc3f4f26706d8
I20260923 22:15:40.337150 17512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:15:40.338397 17512 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:15:40.338627 17512 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Loaded TSK: 0
I20260923 22:15:40.355736 17512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Generated new TSK 1
I20260923 22:15:40.355911 17512 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:15:40.409503 17502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:40.409725 17502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:40.409761 17502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:40.410965 17502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:40.411015 17502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:40.411052 17502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:15:40.412668 17502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:43409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=45969
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:40.412935 17502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:40.413156 17502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:40.418187 17528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.418181 17527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.418187 17530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:40.418664 17502 server_base.cc:1034] running on GCE node
I20260923 22:15:40.418906 17502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:40.419198 17502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:40.420365 17502 hybrid_clock.cc:648] HybridClock initialized: now 1790201740420346 us; error 32 us; skew 500 ppm
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:40.423334 17502 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:15:40.423871 17502 webserver.cc:460] Webserver started at http://127.1.132.129:45969/ using document root <none> and password file <none>
I20260923 22:15:40.424114 17502 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:40.424211 17502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:40.425901 17502 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:15:40.426653 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.426810 17502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:15:40.426898 17502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "277c2b8d4f0145d7be03f31e97a53622"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "367b6f2e7645f3fdf3a14ef385d14699"
server_key_iv: "f32b08e1f8afd0229956fcd1cc49f4ac"
server_key_version: "encryptionkey@0"
I20260923 22:15:40.427327 17502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:40.454190 17502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:40.456568 17502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:40.456789 17502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:40.457062 17502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:40.457561 17544 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:15:40.459080 17502 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:15:40.459146 17502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:15:40.459178 17502 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:15:40.460033 17502 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:15:40.460076 17502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:15:40.460243 17544 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap starting.
I20260923 22:15:40.467127 17502 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:43409
I20260923 22:15:40.467212 17651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:43409 every 8 connection(s)
I20260923 22:15:40.467700 17502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:15:40.470448 17544 log.cc:824] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:40.472476 17544 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:15:40.472715 17544 tablet_bootstrap.cc:492] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap complete.
I20260923 22:15:40.473754 17544 ts_tablet_manager.cc:1397] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260923 22:15:40.474838 17544 raft_consensus.cc:348] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:40.475164 17544 raft_consensus.cc:729] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277c2b8d4f0145d7be03f31e97a53622, State: Initialized, Role: FOLLOWER
I20260923 22:15:40.475348 17544 consensus_queue.cc:260] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } }
I20260923 22:15:40.475610 17544 ts_tablet_manager.cc:1428] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:15:40.476135 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17502
I20260923 22:15:40.476485 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:39171
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=40769
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:40.480645 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.469082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:52979 (local address 127.1.132.190:40719)
0923 22:15:40.469269 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.469278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.469540 (+ 262us) server_negotiation.cc:407] Connection header received
0923 22:15:40.470353 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.470362 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.470429 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.470527 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.471425 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.472793 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.474218 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.474796 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.474904 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.477236 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.477240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.477241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.477253 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.478768 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.479259 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.479262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.479262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.479368 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.479660 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.479661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.479662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.479967 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.480035 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.480394 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.480500 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260923 22:15:40.480744 17654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.469022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:52979)
0923 22:15:40.469355 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.469409 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.470130 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.470698 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.470701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.471045 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:40.471290 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.471296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.472907 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.472908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.474114 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.474117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.474888 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.474889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.475029 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.475634 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:40.475645 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:40.477133 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:40.478853 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.478860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.478862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.479155 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.479438 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.479438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.479438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.479581 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.480038 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:40.480045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:40.480233 (+ 188us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.480356 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.480544 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":116,"threads_started":1}
I20260923 22:15:40.481485 17652 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:40.481585 17652 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:40.481755 17652 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:40.482465 17464 ts_manager.cc:194] Registered new tserver with Master: 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409)
I20260923 22:15:40.483516 17464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:52979
I20260923 22:15:40.484490 17652 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:15:40.492251 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.483083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.0.0.1:55634)
0923 22:15:40.483266 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.483310 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.483380 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.484677 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.484685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.485030 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:40.485095 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.485098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.485957 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.485958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.487015 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.487019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.487440 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.487441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.487519 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.488009 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:40.488014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:40.489566 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:40.491160 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.491167 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.491170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.491358 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.491546 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.491546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.491547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.491609 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.491855 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:40.491862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:40.491922 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.491995 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.492126 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260923 22:15:40.492251 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.483777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.1.132.129:43409)
0923 22:15:40.483932 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.483936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.483944 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:15:40.483974 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.483976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.483989 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.484670 (+ 681us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.485193 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.485866 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.487087 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.487368 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.487429 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.489648 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.489657 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.489657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.489664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.491079 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.491434 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.491437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.491437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.491489 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.491659 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.491659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.491660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.491794 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.491864 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.492037 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.492124 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260923 22:15:40.492766 17586 tablet_service.cc:1514] Processing DeleteTablet for tablet cb7badcb882d41ccaff7444ec76cfa1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:20 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:55634
I20260923 22:15:40.493383 17661 tablet_replica.cc:330] stopping tablet replica
I20260923 22:15:40.493513 17661 raft_consensus.cc:2229] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:40.493587 17661 raft_consensus.cc:2258] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:40.493867 17661 ts_tablet_manager.cc:1905] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:40.495880 17661 ts_tablet_manager.cc:1918] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:40.495950 17661 log.cc:1196] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal/wals/cb7badcb882d41ccaff7444ec76cfa1e
I20260923 22:15:40.496145 17661 ts_tablet_manager.cc:1939] T cb7badcb882d41ccaff7444ec76cfa1e P 277c2b8d4f0145d7be03f31e97a53622: Deleting consensus metadata
I20260923 22:15:40.496663 17448 catalog_manager.cc:4841] TS 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409): tablet cb7badcb882d41ccaff7444ec76cfa1e (table test-table [id=3d64b57f153a4c4eaa2b466a8c350f16]) successfully deleted
W20260923 22:15:40.559696 17658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:40.559916 17658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:40.559939 17658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:40.561169 17658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:40.561213 17658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:40.561239 17658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:15:40.562737 17658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:39171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=40769
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:40.562939 17658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:40.563126 17658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:40.567936 17667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.567965 17666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.567975 17669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:40.568212 17658 server_base.cc:1034] running on GCE node
I20260923 22:15:40.568400 17658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:40.568734 17658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:40.569887 17658 hybrid_clock.cc:648] HybridClock initialized: now 1790201740569879 us; error 33 us; skew 500 ppm
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:40.572381 17658 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:15:40.572854 17658 webserver.cc:460] Webserver started at http://127.1.132.130:40769/ using document root <none> and password file <none>
I20260923 22:15:40.573067 17658 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:40.573124 17658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:40.574482 17658 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.575179 17676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.575416 17658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.575503 17658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "59faf951892d4202a250465bfc2869e3"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "334e8170159ad0b6d2650f8f53282cb0"
server_key_iv: "5791043ca1a0c0082f24f7ff8855c525"
server_key_version: "encryptionkey@0"
I20260923 22:15:40.575903 17658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:40.589818 17658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:40.591602 17658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:40.591735 17658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:40.591949 17658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:40.592278 17658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:40.592315 17658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.592345 17658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:40.592368 17658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.597886 17658 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39171
I20260923 22:15:40.597957 17789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39171 every 8 connection(s)
I20260923 22:15:40.598268 17658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:15:40.601622 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17658
I20260923 22:15:40.601959 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:45605
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=34117
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:40.608465 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.599003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:57295 (local address 127.1.132.190:40719)
0923 22:15:40.599075 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.599077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.599237 (+ 160us) server_negotiation.cc:407] Connection header received
0923 22:15:40.599746 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.599748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.599761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.599826 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.600844 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.601374 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.602773 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.603020 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.603077 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.605548 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.605550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.605550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.605558 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.606954 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.607442 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.607443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.607444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.607504 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.607773 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.607774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.607774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.607878 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.607923 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.608297 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.608386 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:15:40.608712 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.598953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:57295)
0923 22:15:40.599100 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.599140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.599637 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.599907 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.599911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.600536 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:40.600741 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.600746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.601484 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.601486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.602685 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.602689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.603100 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.603101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.603252 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.603894 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:40.603903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:40.605413 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:40.607034 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.607043 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.607045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.607357 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.607570 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.607571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.607571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.607715 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.607934 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:40.607939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:40.608127 (+ 188us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.608280 (+ 153us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.608464 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260923 22:15:40.609229 17790 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:40.609338 17790 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:40.609539 17790 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:40.609933 17464 ts_manager.cc:194] Registered new tserver with Master: 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171)
I20260923 22:15:40.610302 17464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:57295
W20260923 22:15:40.684924 17793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:15:40.685134 17793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:15:40.685159 17793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:15:40.686349 17793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:15:40.686395 17793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:15:40.686421 17793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:15:40.687927 17793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:45605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=34117
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:15:40.688162 17793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:15:40.688410 17793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:15:40.693325 17798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:15:40.693332 17799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:40.693490 17793 server_base.cc:1034] running on GCE node
W20260923 22:15:40.693466 17801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:15:40.693751 17793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:15:40.693969 17793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:15:40.695107 17793 hybrid_clock.cc:648] HybridClock initialized: now 1790201740695094 us; error 30 us; skew 500 ppm
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:15:40.697551 17793 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:15:40.698004 17793 webserver.cc:460] Webserver started at http://127.1.132.131:34117/ using document root <none> and password file <none>
I20260923 22:15:40.698200 17793 fs_manager.cc:359] Metadata directory not provided
I20260923 22:15:40.698256 17793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:15:40.699579 17793 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.700129 17808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.700321 17793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.700404 17793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9164d545640440dd848673f2a76a966d"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "18066dff4713b5e211e1034237493109"
server_key_iv: "3543b4e513f39915c0d16f1324f243ad"
server_key_version: "encryptionkey@0"
I20260923 22:15:40.700723 17793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:15:40.722301 17793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:15:40.724407 17793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:15:40.724524 17793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:15:40.724711 17793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:15:40.724979 17793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:15:40.725008 17793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.725029 17793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:15:40.725039 17793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.730434 17793 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45605
I20260923 22:15:40.730515 17921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45605 every 8 connection(s)
I20260923 22:15:40.730813 17793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:15:40 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201740, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:15:40.737219 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17793
I20260923 22:15:40.740283 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.731482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:60077 (local address 127.1.132.190:40719)
0923 22:15:40.731537 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.731539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.731728 (+ 189us) server_negotiation.cc:407] Connection header received
0923 22:15:40.732327 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.732329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.732349 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.732395 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.733308 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.733796 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.734744 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.734960 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.735016 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.737234 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.737235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.737236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.737245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.738843 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.739334 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.739335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.739335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.739403 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.739632 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.739633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.739634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.739762 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.739827 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.740086 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.740166 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:15:40.740398 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.731485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:60077)
0923 22:15:40.731617 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.731652 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.732229 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.732472 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.732476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.733019 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:15:40.733216 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.733221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.733879 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.733880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.734665 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.734667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.735029 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.735029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.735203 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.735638 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:15:40.735646 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:15:40.737127 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:15:40.738916 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.738958 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.738961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.739257 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.739469 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:15:40.739469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:15:40.739470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:15:40.739580 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:15:40.739818 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:15:40.739821 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:15:40.739968 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.740049 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.740170 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260923 22:15:40.740947 17922 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40719
I20260923 22:15:40.741055 17922 heartbeater.cc:464] Registering TS with master...
I20260923 22:15:40.741242 17922 heartbeater.cc:511] Master 127.1.132.190:40719 requested a full tablet report, sending...
I20260923 22:15:40.741678 17464 ts_manager.cc:194] Registered new tserver with Master: 9164d545640440dd848673f2a76a966d (127.1.132.131:45605)
I20260923 22:15:40.742106 17464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:60077
I20260923 22:15:40.744104 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.737603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.1.132.190:40719)
0923 22:15:40.737791 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.737793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.737833 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:15:40.737976 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.737977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.737988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.738037 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.738720 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.739384 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.740352 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.740733 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.740787 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.741982 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.741983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.741983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.741989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.743263 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.743543 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.743544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.743544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.743592 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.743714 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.743715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.743715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.743797 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.743901 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.743968 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.744032 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":58,"threads_started":1}
I20260923 22:15:40.744776 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:15:40.747236 17392 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.132.190:40719)
I20260923 22:15:40.747323 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.745063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.1.132.190:40719)
0923 22:15:40.745109 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.745111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.745184 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:15:40.745233 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.745233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.745260 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.745316 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:40.745395 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.745835 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.746562 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.746748 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.746807 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.746933 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:40.746977 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 22:15:40.747265 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41128: token expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 22:15:40.747365 17925 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41128: token expired
I20260923 22:15:40.753926 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.747455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.1.132.190:40719)
0923 22:15:40.747488 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.747489 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.748075 (+ 586us) server_negotiation.cc:407] Connection header received
0923 22:15:40.748129 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.748130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.748140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.748238 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:15:40.748648 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.749288 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.750288 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.750471 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.750520 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.751581 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:15:40.751582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:15:40.751583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:15:40.751590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:15:40.752934 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.753239 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.753240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.753240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.753294 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:15:40.753462 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:15:40.753462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:15:40.753463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:15:40.753570 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:15:40.753655 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.753750 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.753832 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:15:40.755105 17464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:15:40.756702 17464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:15:40.759558 17586 tablet_service.cc:1467] Processing CreateTablet for tablet 255dbb2190684bbdb54920e474784a55 (DEFAULT_TABLE table=test-table [id=970169b650e7485da92d0e6000fd7ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:40.759778 17586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255dbb2190684bbdb54920e474784a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:40.761265 17544 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap starting.
I20260923 22:15:40.761610 17544 tablet_bootstrap.cc:654] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:40.762511 17544 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: No bootstrap required, opened a new log
I20260923 22:15:40.762567 17544 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:15:40.762717 17544 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.762794 17544 raft_consensus.cc:374] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:40.762848 17544 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277c2b8d4f0145d7be03f31e97a53622, State: Initialized, Role: FOLLOWER
I20260923 22:15:40.762892 17544 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.762982 17544 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:15:40.763597 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.759215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39171 (local address 127.0.0.1:48584)
0923 22:15:40.759733 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.759743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.759806 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.760059 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.760060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.760062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:40.760119 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.760121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.761117 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.761118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.762182 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.762185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.763154 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.763155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.763226 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.763369 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.763445 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.763512 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":31,"thread_start_us":105,"threads_started":1}
I20260923 22:15:40.763837 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.759454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.1.132.130:39171)
0923 22:15:40.759728 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.759732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.759792 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:15:40.759921 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.759923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.759939 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.760043 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:40.760223 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.761001 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.762282 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.763043 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.763120 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.763188 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.763602 (+ 414us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.763656 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":160,"threads_started":1}
I20260923 22:15:40.764541 17724 tablet_service.cc:1467] Processing CreateTablet for tablet 255dbb2190684bbdb54920e474784a55 (DEFAULT_TABLE table=test-table [id=970169b650e7485da92d0e6000fd7ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:40.764955 17724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255dbb2190684bbdb54920e474784a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:40.765079 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.759215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45605 (local address 127.0.0.1:32902)
0923 22:15:40.759312 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.759324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.759362 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.761523 (+ 2161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.761524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.761526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:40.761562 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.761563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.762729 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.762730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.763887 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.763890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.764810 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.764811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.764886 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.764889 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.764945 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.764993 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260923 22:15:40.765095 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.760285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.1.132.131:45605)
0923 22:15:40.761351 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.761355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.761366 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:15:40.761421 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.761423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.761443 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.761527 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:40.761622 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.762639 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.763996 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.764676 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.764743 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.764807 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.764965 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.765010 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":83,"threads_started":1}
I20260923 22:15:40.765996 17856 tablet_service.cc:1467] Processing CreateTablet for tablet 255dbb2190684bbdb54920e474784a55 (DEFAULT_TABLE table=test-table [id=970169b650e7485da92d0e6000fd7ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:15:40.766289 17856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255dbb2190684bbdb54920e474784a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:15:40.768047 17932 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Bootstrap starting.
I20260923 22:15:40.768647 17932 tablet_bootstrap.cc:654] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:40.769126 17933 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Bootstrap starting.
I20260923 22:15:40.769183 17932 log.cc:824] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:40.769752 17933 tablet_bootstrap.cc:654] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Neither blocks nor log segments found. Creating new log.
I20260923 22:15:40.769958 17932 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: No bootstrap required, opened a new log
I20260923 22:15:40.770030 17932 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:15:40.770331 17933 log.cc:824] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:15:40.771044 17933 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: No bootstrap required, opened a new log
I20260923 22:15:40.771111 17933 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:15:40.772004 17932 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.772214 17932 raft_consensus.cc:374] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:40.772261 17932 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59faf951892d4202a250465bfc2869e3, State: Initialized, Role: FOLLOWER
I20260923 22:15:40.772372 17932 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.772670 17932 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260923 22:15:40.772689 17790 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
I20260923 22:15:40.773121 17933 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.773272 17933 raft_consensus.cc:374] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:15:40.773324 17933 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9164d545640440dd848673f2a76a966d, State: Initialized, Role: FOLLOWER
I20260923 22:15:40.773432 17933 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.773698 17933 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:15:40.773695 17922 heartbeater.cc:503] Master 127.1.132.190:40719 was elected leader, sending a full tablet report...
I20260923 22:15:40.845991 17936 raft_consensus.cc:482] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:15:40.846123 17936 raft_consensus.cc:504] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.846520 17936 leader_election.cc:288] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9164d545640440dd848673f2a76a966d (127.1.132.131:45605), 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409)
W20260923 22:15:40.848816 17791 tablet.cc:2369] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:15:40.849479 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.846777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:32851 (local address 127.1.132.131:45605)
0923 22:15:40.846824 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.846825 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.846835 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:40.846920 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.846920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.846928 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.847022 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:40.847163 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.847631 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.848618 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.849170 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.849239 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.849248 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.849379 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.849417 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:15:40.849511 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.846768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:54423 (local address 127.1.132.129:43409)
0923 22:15:40.846817 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.846819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.847005 (+ 186us) server_negotiation.cc:407] Connection header received
0923 22:15:40.847039 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.847040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.847066 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.847122 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:15:40.847263 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.847738 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.848687 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.849177 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.849247 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.849332 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.849398 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.849440 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:15:40.849530 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.846726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45605 (local address 127.1.132.130:32851)
0923 22:15:40.846787 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.846799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.846846 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.847025 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.847026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.847028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:40.847101 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.847103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.847715 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.847716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.848528 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.848531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.849246 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.849246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.849319 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.849323 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.849368 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.849411 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260923 22:15:40.849687 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.846727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.1.132.130:54423)
0923 22:15:40.846941 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.846948 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:15:40.846989 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:15:40.847130 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:15:40.847131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:15:40.847132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:15:40.847199 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.847200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.847812 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.847813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.848605 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:15:40.848607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.849256 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.849257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:15:40.849333 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.849337 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:15:40.849388 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:15:40.849432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":15,"thread_start_us":138,"threads_started":1}
I20260923 22:15:40.849860 17876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "59faf951892d4202a250465bfc2869e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9164d545640440dd848673f2a76a966d" is_pre_election: true
I20260923 22:15:40.850028 17876 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59faf951892d4202a250465bfc2869e3 in term 0.
I20260923 22:15:40.850032 17606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "59faf951892d4202a250465bfc2869e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277c2b8d4f0145d7be03f31e97a53622" is_pre_election: true
I20260923 22:15:40.850185 17606 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59faf951892d4202a250465bfc2869e3 in term 0.
I20260923 22:15:40.850270 17678 leader_election.cc:302] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59faf951892d4202a250465bfc2869e3, 9164d545640440dd848673f2a76a966d; no voters:
I20260923 22:15:40.850389 17936 raft_consensus.cc:2781] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:15:40.850443 17936 raft_consensus.cc:482] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:15:40.850463 17936 raft_consensus.cc:3037] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:40.851152 17936 raft_consensus.cc:504] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.851274 17936 leader_election.cc:288] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [CANDIDATE]: Term 1 election: Requested vote from peers 9164d545640440dd848673f2a76a966d (127.1.132.131:45605), 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409)
I20260923 22:15:40.851416 17876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "59faf951892d4202a250465bfc2869e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9164d545640440dd848673f2a76a966d"
I20260923 22:15:40.851478 17606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "59faf951892d4202a250465bfc2869e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277c2b8d4f0145d7be03f31e97a53622"
I20260923 22:15:40.851503 17876 raft_consensus.cc:3037] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:40.851557 17606 raft_consensus.cc:3037] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:15:40.852399 17876 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59faf951892d4202a250465bfc2869e3 in term 1.
I20260923 22:15:40.852399 17606 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59faf951892d4202a250465bfc2869e3 in term 1.
I20260923 22:15:40.852551 17678 leader_election.cc:302] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59faf951892d4202a250465bfc2869e3, 9164d545640440dd848673f2a76a966d; no voters:
I20260923 22:15:40.852630 17936 raft_consensus.cc:2781] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:15:40.852766 17936 raft_consensus.cc:686] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 LEADER]: Becoming Leader. State: Replica: 59faf951892d4202a250465bfc2869e3, State: Running, Role: LEADER
I20260923 22:15:40.852856 17936 consensus_queue.cc:237] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:15:40.853809 17464 catalog_manager.cc:5495] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59faf951892d4202a250465bfc2869e3 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "59faf951892d4202a250465bfc2869e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } health_report { overall_health: UNKNOWN } } }
I20260923 22:15:40.879663 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.877436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.1.132.130:39171)
0923 22:15:40.877499 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:15:40.877501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:15:40.877511 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:15:40.877548 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:15:40.877549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:15:40.877567 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:15:40.877626 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:15:40.877742 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.878249 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.878975 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:15:40.879182 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:15:40.879245 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:15:40.879369 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:15:40.879470 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:15:40.879529 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:15:40.879543 (+ 14us) server_negotiation.cc:299] Negotiation successful
0923 22:15:40.879560 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:15:40.881349 17876 raft_consensus.cc:1261] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Refusing update from remote peer 59faf951892d4202a250465bfc2869e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:40.881384 17606 raft_consensus.cc:1261] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Refusing update from remote peer 59faf951892d4202a250465bfc2869e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:15:40.881690 17939 consensus_queue.cc:1035] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:40.881824 17936 consensus_queue.cc:1035] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:15:40.884125 17942 mvcc.cc:205] Tried to move back new op lower bound from 7332666330647625728 to 7332666330534342656. Current Snapshot: MvccSnapshot[applied={T|T < 7332666330647625728}]
I20260923 22:15:40.884156 17944 mvcc.cc:205] Tried to move back new op lower bound from 7332666330647625728 to 7332666330534342656. Current Snapshot: MvccSnapshot[applied={T|T < 7332666330647625728}]
I20260923 22:15:40.884268 17943 mvcc.cc:205] Tried to move back new op lower bound from 7332666330647625728 to 7332666330534342656. Current Snapshot: MvccSnapshot[applied={T|T < 7332666330647625728}]
I20260923 22:15:40.886161 17463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:40.886245 17463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:15:40.886970 17463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Sending DeleteTablet for 3 replicas of tablet 255dbb2190684bbdb54920e474784a55
I20260923 22:15:40.887234 17586 tablet_service.cc:1514] Processing DeleteTablet for tablet 255dbb2190684bbdb54920e474784a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:40 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:55634
I20260923 22:15:40.887265 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17502
I20260923 22:15:40.887283 17856 tablet_service.cc:1514] Processing DeleteTablet for tablet 255dbb2190684bbdb54920e474784a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:40 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:32902
I20260923 22:15:40.887440 17661 tablet_replica.cc:330] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: stopping tablet replica
I20260923 22:15:40.887490 17661 raft_consensus.cc:2229] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:40.887538 17661 raft_consensus.cc:2258] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:40.888052 17724 tablet_service.cc:1514] Processing DeleteTablet for tablet 255dbb2190684bbdb54920e474784a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:15:40 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:48584
I20260923 22:15:40.888309 17952 tablet_replica.cc:330] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: stopping tablet replica
I20260923 22:15:40.888401 17952 raft_consensus.cc:2229] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:15:40.888484 17952 raft_consensus.cc:2258] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:40.889166 17952 ts_tablet_manager.cc:1905] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:15:40.890808 17952 ts_tablet_manager.cc:1918] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:15:40.890903 17952 log.cc:1196] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/255dbb2190684bbdb54920e474784a55
I20260923 22:15:40.891165 17952 ts_tablet_manager.cc:1939] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Deleting consensus metadata
I20260923 22:15:40.891546 17449 catalog_manager.cc:4841] TS 9164d545640440dd848673f2a76a966d (127.1.132.131:45605): tablet 255dbb2190684bbdb54920e474784a55 (table test-table [id=970169b650e7485da92d0e6000fd7ab9]) successfully deleted
I20260923 22:15:40.892289 17953 tablet_replica.cc:330] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: stopping tablet replica
I20260923 22:15:40.892403 17953 raft_consensus.cc:2229] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:15:40.892529 17953 raft_consensus.cc:2258] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:15:40.892874 17953 ts_tablet_manager.cc:1905] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:15:40.893530 17448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:43409 (error 108)
I20260923 22:15:40.893893 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17658
I20260923 22:15:40.894013 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.893880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.0.0.1:55644)
0923 22:15:40.893939 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.893983 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 22:15:40.894198 17448 catalog_manager.cc:4584] TS 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 255dbb2190684bbdb54920e474784a55: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43409: connect: Connection refused (error 111)
I20260923 22:15:40.899112 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17793
I20260923 22:15:40.899233 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:15:40.899113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39171 (local address 127.0.0.1:48596)
0923 22:15:40.899164 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:15:40.899194 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.130:39171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:15:40.902755 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 17431
I20260923 22:16:00.907341 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--webserver_interface=127.1.132.190
--webserver_port=40795
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:00.989156 17955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:00.989938 17955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:00.990049 17955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:00.992384 17955 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:16:00.992462 17955 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:16:00.992483 17955 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:16:00.992501 17955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:00.992520 17955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:00.992547 17955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:00.992563 17955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:00.992581 17955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:00.994542 17955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=40795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:00.994829 17955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:00.995064 17955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:01.000555 17955 server_base.cc:1034] running on GCE node
W20260923 22:16:01.000526 17960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.000566 17961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.000526 17963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.001044 17955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:01.001343 17955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:01.002496 17955 hybrid_clock.cc:648] HybridClock initialized: now 1790201761002485 us; error 33 us; skew 500 ppm
Sep 23 22:16:01 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:01.010782 17955 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:01.011317 17955 webserver.cc:460] Webserver started at http://127.1.132.190:40795/ using document root <none> and password file <none>
I20260923 22:16:01.011612 17955 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:01.011684 17955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:01.013290 17955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:01.014104 17970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.014305 17955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:01.014410 17955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "565f1b0c876941c3ae906d180a0d67fa"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "101b4394f10c439f9f2161844c9d8803"
server_key_iv: "a47afa84a41d4d25f6d51bd18940138d"
server_key_version: "encryptionkey@0"
I20260923 22:16:01.014828 17955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:01.029628 17955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:01.032014 17955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:01.032138 17955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:01.036170 17955 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40719
I20260923 22:16:01.036244 18022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40719 every 8 connection(s)
I20260923 22:16:01.036623 17955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:01.038221 18023 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:16:01.039034 18023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap starting.
I20260923 22:16:01.042579 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 17955
I20260923 22:16:01.042914 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:43409
--local_ip_for_outbound_sockets=127.1.132.129
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=45969
--webserver_interface=127.1.132.129
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:01.048624 18023 log.cc:824] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:01.050880 18023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:16:01.051184 18023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Bootstrap complete.
I20260923 22:16:01.053581 18023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:16:01.053887 18023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Initialized, Role: FOLLOWER
I20260923 22:16:01.054073 18023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:16:01.054155 18023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:01.054206 18023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:01.054266 18023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:16:01.055379 18023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:16:01.055485 18023 leader_election.cc:302] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 565f1b0c876941c3ae906d180a0d67fa; no voters:
I20260923 22:16:01.055655 18023 leader_election.cc:288] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:16:01.055742 18027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:16:01.055924 18023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:01.055981 18027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [term 3 LEADER]: Becoming Leader. State: Replica: 565f1b0c876941c3ae906d180a0d67fa, State: Running, Role: LEADER
I20260923 22:16:01.056083 18027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } }
I20260923 22:16:01.056407 18027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565f1b0c876941c3ae906d180a0d67fa. Latest consensus state: current_term: 3 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:16:01.056486 18027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:01.056440 18028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "565f1b0c876941c3ae906d180a0d67fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565f1b0c876941c3ae906d180a0d67fa" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40719 } } }
I20260923 22:16:01.056913 18028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:01.056833 18036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:01.057964 18036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:01.058063 18036 catalog_manager.cc:1259] Loaded cluster ID: 37ab16b30eaa4237ac9dc3f4f26706d8
I20260923 22:16:01.058084 18036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:01.059473 18036 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:01.059666 18036 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Loaded TSK: 0
I20260923 22:16:01.059707 18036 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 565f1b0c876941c3ae906d180a0d67fa: Loaded TSK: 1
F20260923 22:16:01.059953 18036 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f82274fa9ad google::LogMessage::Fail() at ??:0
@ 0x7f82274fd88a google::LogMessage::SendToLog() at ??:0
@ 0x7f82274fa4de google::LogMessage::Flush() at ??:0
@ 0x7f82274fba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8229c67104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8229c64d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f822f4581cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f822f45848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f822f458594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f822f41ec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f822f428ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f82293c80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f82293bd954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f822a2216db start_thread at ??:0
@ 0x7f8225ff861f clone at ??:0
W20260923 22:16:01.145179 18025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:01.145381 18025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:01.145406 18025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:01.146605 18025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:01.146652 18025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:01.146678 18025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:01.148545 18025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:43409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=45969
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:01.148810 18025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:01.149046 18025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:01.154238 18064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.154392 18025 server_base.cc:1034] running on GCE node
W20260923 22:16:01.154238 18067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.154261 18065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.154743 18025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:01.154974 18025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:01.156138 18025 hybrid_clock.cc:648] HybridClock initialized: now 1790201761156125 us; error 30 us; skew 500 ppm
Sep 23 22:16:01 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:01.158591 18025 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:16:01.159011 18025 webserver.cc:460] Webserver started at http://127.1.132.129:45969/ using document root <none> and password file <none>
I20260923 22:16:01.159226 18025 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:01.159276 18025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:01.160658 18025 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:16:01.161324 18074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.161479 18025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:16:01.161594 18025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "277c2b8d4f0145d7be03f31e97a53622"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "367b6f2e7645f3fdf3a14ef385d14699"
server_key_iv: "f32b08e1f8afd0229956fcd1cc49f4ac"
server_key_version: "encryptionkey@0"
I20260923 22:16:01.161934 18025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:01.177795 18025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:01.179950 18025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:01.180087 18025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:01.180346 18025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:01.180786 18081 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:16:01.182777 18025 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:16:01.182842 18025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:16:01.182880 18025 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:16:01.183574 18025 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:16:01.183610 18025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:16:01.183717 18081 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap starting.
I20260923 22:16:01.191464 18025 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:43409
I20260923 22:16:01.191731 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:43409 every 8 connection(s)
I20260923 22:16:01.192155 18025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:01.195478 18081 log.cc:824] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:01.198531 18081 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:16:01.198905 18081 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Bootstrap complete.
I20260923 22:16:01.198913 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18025
I20260923 22:16:01.199179 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:39171
--local_ip_for_outbound_sockets=127.1.132.130
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=40769
--webserver_interface=127.1.132.130
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:01.200014 18081 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.012s
I20260923 22:16:01.201654 18081 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:01.202023 18081 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277c2b8d4f0145d7be03f31e97a53622, State: Initialized, Role: FOLLOWER
I20260923 22:16:01.202224 18081 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:01.202534 18081 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260923 22:16:01.259195 18191 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:01.193165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:43953)
0923 22:16:01.193345 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.193391 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:01.193945 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:01.258987 (+ 65042us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
W20260923 22:16:01.259467 18189 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40719 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:40719: Client connection negotiation failed: client connection to 127.1.132.190:40719: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:16:01.259833 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.259711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:52467)
0923 22:16:01.259765 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.259808 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:16:01.260147 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.260072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:46143)
0923 22:16:01.260106 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.260130 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:16:01.260252 18189 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:16:01.286379 18193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:01.286665 18193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:01.286693 18193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:01.288111 18193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:01.288230 18193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:01.288277 18193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:01.289932 18193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:39171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=40769
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:01.290153 18193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:01.290352 18193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:01.295615 18199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.295619 18200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.295678 18193 server_base.cc:1034] running on GCE node
W20260923 22:16:01.295615 18202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.296080 18193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:01.296372 18193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:01.297536 18193 hybrid_clock.cc:648] HybridClock initialized: now 1790201761297513 us; error 37 us; skew 500 ppm
Sep 23 22:16:01 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:01.300127 18193 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:16:01.300580 18193 webserver.cc:460] Webserver started at http://127.1.132.130:40769/ using document root <none> and password file <none>
I20260923 22:16:01.300782 18193 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:01.300832 18193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:01.302188 18193 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:01.303054 18209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.303261 18193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:01.303346 18193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "59faf951892d4202a250465bfc2869e3"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "334e8170159ad0b6d2650f8f53282cb0"
server_key_iv: "5791043ca1a0c0082f24f7ff8855c525"
server_key_version: "encryptionkey@0"
I20260923 22:16:01.303697 18193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:01.321724 18193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:01.323865 18193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:01.324007 18193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:01.324277 18193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:01.324791 18216 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:16:01.326256 18193 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:16:01.326318 18193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:16:01.326356 18193 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:16:01.327047 18193 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:16:01.327081 18193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:16:01.327178 18216 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Bootstrap starting.
I20260923 22:16:01.328931 18216 log.cc:824] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:01.332168 18216 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:16:01.332557 18216 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Bootstrap complete.
I20260923 22:16:01.332719 18216 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260923 22:16:01.333799 18193 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:39171
I20260923 22:16:01.333860 18324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:39171 every 8 connection(s)
I20260923 22:16:01.334180 18193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:01.334662 18216 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:01.334981 18216 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59faf951892d4202a250465bfc2869e3, State: Initialized, Role: FOLLOWER
I20260923 22:16:01.335135 18216 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:01.335351 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.334966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:36979)
0923 22:16:01.335126 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.335208 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260923 22:16:01.335381 18216 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260923 22:16:01.335528 18325 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40719 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:40719: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
I20260923 22:16:01.335752 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18193
I20260923 22:16:01.335788 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.335702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:39173)
0923 22:16:01.335740 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.335764 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:16:01.336004 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:45605
--local_ip_for_outbound_sockets=127.1.132.131
--tserver_master_addrs=127.1.132.190:40719
--webserver_port=34117
--webserver_interface=127.1.132.131
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:01.336097 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.336021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:59831)
0923 22:16:01.336055 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.336078 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:16:01.336201 18325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:16:01.417588 18329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:01.417811 18329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:01.417835 18329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:01.418977 18329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:01.419011 18329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:01.419036 18329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:01.420469 18329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:45605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=34117
--tserver_master_addrs=127.1.132.190:40719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:01.420670 18329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:01.420853 18329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:01.425752 18329 server_base.cc:1034] running on GCE node
W20260923 22:16:01.425725 18335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.425732 18337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:01.425952 18334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:01.426187 18329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:01.426425 18329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:01.427572 18329 hybrid_clock.cc:648] HybridClock initialized: now 1790201761427557 us; error 32 us; skew 500 ppm
Sep 23 22:16:01 dist-test-slave-l4q5 krb5kdc[16883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:01.430070 18329 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:16:01.430508 18329 webserver.cc:460] Webserver started at http://127.1.132.131:34117/ using document root <none> and password file <none>
I20260923 22:16:01.430722 18329 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:01.430774 18329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:01.432137 18329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:01.432745 18344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.432899 18329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.432978 18329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "9164d545640440dd848673f2a76a966d"
format_stamp: "Formatted at 2026-09-23 22:15:19 on dist-test-slave-l4q5"
server_key: "18066dff4713b5e211e1034237493109"
server_key_iv: "3543b4e513f39915c0d16f1324f243ad"
server_key_version: "encryptionkey@0"
I20260923 22:16:01.433326 18329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:01.445732 18329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:01.447599 18329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:01.447734 18329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:01.447958 18329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:01.448336 18329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:01.448379 18329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.448410 18329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:01.448431 18329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:01.454058 18329 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:45605
I20260923 22:16:01.454109 18457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:45605 every 8 connection(s)
I20260923 22:16:01.454444 18329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:01.455473 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.455126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:57951)
0923 22:16:01.455272 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.455339 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
W20260923 22:16:01.455873 18458 heartbeater.cc:644] Failed to heartbeat to 127.1.132.190:40719 (0 consecutive failures): Network error: Failed to ping master at 127.1.132.190:40719: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
I20260923 22:16:01.456151 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.456056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:52093)
0923 22:16:01.456108 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.456133 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:16:01.456420 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:01.456341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:44241)
0923 22:16:01.456385 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:01.456405 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:16:01.456527 18458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:16:01.460857 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18329
I20260923 22:16:02.260871 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.260606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:44179)
0923 22:16:02.260763 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.260809 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260923 22:16:02.336746 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.336515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:51669)
0923 22:16:02.336666 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.336701 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260923 22:16:02.448758 18464 raft_consensus.cc:482] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:02.448902 18464 raft_consensus.cc:504] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:02.449239 18464 leader_election.cc:288] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171), 9164d545640440dd848673f2a76a966d (127.1.132.131:45605)
Sep 23 22:16:02 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.131@KRBTEST.COM
Sep 23 22:16:02 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.130@KRBTEST.COM
I20260923 22:16:02.457209 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.456840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:37731)
0923 22:16:02.457122 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.457158 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":69,"threads_started":1}
I20260923 22:16:02.459884 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.449482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:45605 (local address 127.1.132.129:54941)
0923 22:16:02.449573 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.449582 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:02.449622 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:02.450490 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:02.450492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:02.450814 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:02.451010 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:02.451017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.452158 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.452159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:02.452937 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:02.452940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.453530 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:02.453593 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:02.454160 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:02.454190 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:02.455721 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:02.458052 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:02.458062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:02.458300 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:02.458635 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:02.458636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:02.458813 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:02.459350 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:02.459359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:02.459499 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 22:16:02.459595 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:16:02.459774 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":46,"thread_start_us":50,"threads_started":1}
I20260923 22:16:02.459903 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.449567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:54941 (local address 127.1.132.131:45605)
0923 22:16:02.449721 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:02.449731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:02.449751 (+ 20us) server_negotiation.cc:407] Connection header received
0923 22:16:02.450269 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:02.450271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:02.450323 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:02.450450 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:02.451188 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.452056 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453033 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.453456 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453567 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:02.455829 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:02.455835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:02.455835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:02.455847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:02.457963 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458385 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:02.458389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:02.458389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:02.458512 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458918 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:02.458919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:02.458919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:02.459265 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:02.459327 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:02.459665 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 22:16:02.459800 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":41,"threads_started":1}
I20260923 22:16:02.460078 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.449618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:55603 (local address 127.1.132.130:39171)
0923 22:16:02.449931 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:02.449938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:02.449958 (+ 20us) server_negotiation.cc:407] Connection header received
0923 22:16:02.450074 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:02.450076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:02.450117 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:02.450260 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:02.451145 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.452021 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453015 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.453397 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453494 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:02.456158 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:02.456162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:02.456163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:02.456175 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:02.458554 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458976 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:02.458979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:02.458979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:02.459112 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:02.459361 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:02.459362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:02.459362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:02.459583 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:02.459660 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:02.459823 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:16:02.459945 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":56,"threads_started":1}
I20260923 22:16:02.459952 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.449431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:39171 (local address 127.1.132.129:55603)
0923 22:16:02.449508 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.449519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:02.449576 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:02.450320 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:02.450324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:02.450804 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:02.451010 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:02.451017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.452120 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.452121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:02.452934 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:02.452936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:02.453473 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:02.453474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:02.453587 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:02.454228 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:02.454233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:02.456040 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:02.458635 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:02.458636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:02.458636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:02.458888 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:02.459201 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:02.459202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:02.459202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:02.459276 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:02.459653 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:02.459654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:02.459697 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:16:02.459765 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:16:02.459843 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":19}
I20260923 22:16:02.460682 18279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "277c2b8d4f0145d7be03f31e97a53622" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "59faf951892d4202a250465bfc2869e3" is_pre_election: true
I20260923 22:16:02.460850 18279 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 277c2b8d4f0145d7be03f31e97a53622 in term 1.
W20260923 22:16:02.461117 18076 leader_election.cc:341] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9164d545640440dd848673f2a76a966d (127.1.132.131:45605): Not found: Tablet not found: 255dbb2190684bbdb54920e474784a55
I20260923 22:16:02.461238 18078 leader_election.cc:302] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 277c2b8d4f0145d7be03f31e97a53622, 59faf951892d4202a250465bfc2869e3; no voters: 9164d545640440dd848673f2a76a966d
I20260923 22:16:02.461403 18464 raft_consensus.cc:2781] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:16:02.461460 18464 raft_consensus.cc:482] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:02.461475 18464 raft_consensus.cc:3037] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:16:02.462615 18464 raft_consensus.cc:504] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:02.462752 18464 leader_election.cc:288] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 election: Requested vote from peers 59faf951892d4202a250465bfc2869e3 (127.1.132.130:39171), 9164d545640440dd848673f2a76a966d (127.1.132.131:45605)
I20260923 22:16:02.462956 18279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255dbb2190684bbdb54920e474784a55" candidate_uuid: "277c2b8d4f0145d7be03f31e97a53622" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "59faf951892d4202a250465bfc2869e3"
I20260923 22:16:02.463056 18279 raft_consensus.cc:3037] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 1 FOLLOWER]: Advancing to term 2
W20260923 22:16:02.463186 18076 leader_election.cc:341] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9164d545640440dd848673f2a76a966d (127.1.132.131:45605): Not found: Tablet not found: 255dbb2190684bbdb54920e474784a55
I20260923 22:16:02.464332 18279 raft_consensus.cc:2454] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 277c2b8d4f0145d7be03f31e97a53622 in term 2.
I20260923 22:16:02.464505 18078 leader_election.cc:302] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 277c2b8d4f0145d7be03f31e97a53622, 59faf951892d4202a250465bfc2869e3; no voters: 9164d545640440dd848673f2a76a966d
I20260923 22:16:02.464629 18464 raft_consensus.cc:2781] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:16:02.464774 18464 raft_consensus.cc:686] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [term 2 LEADER]: Becoming Leader. State: Replica: 277c2b8d4f0145d7be03f31e97a53622, State: Running, Role: LEADER
I20260923 22:16:02.464861 18464 consensus_queue.cc:237] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:02.465137 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.465030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:36481)
0923 22:16:02.465080 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.465107 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:16:02.912528 18076 consensus_peers.cc:487] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 -> Peer 9164d545640440dd848673f2a76a966d (127.1.132.131:45605): Couldn't send request to peer 9164d545640440dd848673f2a76a966d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 255dbb2190684bbdb54920e474784a55. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:16:02.937045 18279 raft_consensus.cc:1261] T 255dbb2190684bbdb54920e474784a55 P 59faf951892d4202a250465bfc2869e3 [term 2 FOLLOWER]: Refusing update from remote peer 277c2b8d4f0145d7be03f31e97a53622: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260923 22:16:02.937461 18464 consensus_queue.cc:1035] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260923 22:16:02.938370 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.938200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:37793)
0923 22:16:02.938287 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.938325 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":45,"threads_started":1}
I20260923 22:16:02.939076 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.938973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:49895)
0923 22:16:02.939017 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.939044 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:16:02.939255 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:02.939156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:58233)
0923 22:16:02.939202 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:02.939231 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:16:03.372920 18476 ts_tablet_manager.cc:927] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Initiating tablet copy from peer 277c2b8d4f0145d7be03f31e97a53622 (127.1.132.129:43409)
I20260923 22:16:03.373117 18476 tablet_copy_client.cc:316] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: tablet copy: Beginning tablet copy session from remote peer at address 127.1.132.129:43409
Sep 23 22:16:03 dist-test-slave-l4q5 krb5kdc[16883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.129@KRBTEST.COM
I20260923 22:16:03.380813 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.373512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:60291 (local address 127.1.132.129:43409)
0923 22:16:03.373621 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:03.373626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:03.373635 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:03.373659 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:03.373661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:03.373674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:03.373744 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:03.374250 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:03.374971 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:03.375759 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:03.376084 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:03.376141 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:03.378189 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:03.378196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:03.378197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:03.378203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:03.379683 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:03.380052 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:03.380054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:03.380054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:03.380120 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:03.380313 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:03.380314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:03.380315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:03.380467 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:03.380531 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:03.380681 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:16:03.380744 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
I20260923 22:16:03.380825 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.373368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43409 (local address 127.1.132.131:60291)
0923 22:16:03.373508 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.373527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:03.373593 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:03.373780 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:03.373787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:03.374089 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:03.374165 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:03.374167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:03.375074 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:03.375075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:03.375679 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:03.375682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:03.376159 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:03.376160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:03.376270 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:03.376715 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:03.376720 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:03.378114 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:03.379768 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:03.379775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:03.379777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:03.379979 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:03.380170 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:03.380170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:03.380170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:03.380258 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:03.380535 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:03.380541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:03.380587 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:16:03.380650 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:16:03.380745 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260923 22:16:03.381284 18163 tablet_copy_service.cc:140] P 277c2b8d4f0145d7be03f31e97a53622: Received BeginTabletCopySession request for tablet 255dbb2190684bbdb54920e474784a55 from peer 9164d545640440dd848673f2a76a966d ({username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:60291)
I20260923 22:16:03.381357 18163 tablet_copy_service.cc:161] P 277c2b8d4f0145d7be03f31e97a53622: Beginning new tablet copy session on tablet 255dbb2190684bbdb54920e474784a55 from peer 9164d545640440dd848673f2a76a966d at {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:60291: session id = 9164d545640440dd848673f2a76a966d-255dbb2190684bbdb54920e474784a55
I20260923 22:16:03.381880 18163 tablet_copy_source_session.cc:205] T 255dbb2190684bbdb54920e474784a55 P 277c2b8d4f0145d7be03f31e97a53622: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:16:03.382823 18476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255dbb2190684bbdb54920e474784a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:03.385429 18476 tablet_copy_client.cc:791] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: tablet copy: Starting download of 0 data blocks...
I20260923 22:16:03.385577 18476 tablet_copy_client.cc:655] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: tablet copy: Starting download of 1 WAL segments...
I20260923 22:16:03.386445 18476 tablet_copy_client.cc:531] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:16:03.387173 18476 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Bootstrap starting.
I20260923 22:16:03.388571 18476 log.cc:824] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:03.390849 18476 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:16:03.391050 18476 tablet_bootstrap.cc:492] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Bootstrap complete.
I20260923 22:16:03.391158 18476 ts_tablet_manager.cc:1397] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:16:03.392105 18476 raft_consensus.cc:348] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:03.392297 18476 raft_consensus.cc:729] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9164d545640440dd848673f2a76a966d, State: Initialized, Role: FOLLOWER
I20260923 22:16:03.392432 18476 consensus_queue.cc:260] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59faf951892d4202a250465bfc2869e3" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 39171 } } peers { permanent_uuid: "9164d545640440dd848673f2a76a966d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 45605 } } peers { permanent_uuid: "277c2b8d4f0145d7be03f31e97a53622" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43409 } }
I20260923 22:16:03.392678 18476 ts_tablet_manager.cc:1428] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20260923 22:16:03.393000 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.392885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:36189)
0923 22:16:03.392934 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.392971 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:16:03.392977 18163 tablet_copy_service.cc:342] P 277c2b8d4f0145d7be03f31e97a53622: Request end of tablet copy session 9164d545640440dd848673f2a76a966d-255dbb2190684bbdb54920e474784a55 received from {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:60291
I20260923 22:16:03.393038 18163 tablet_copy_service.cc:433] P 277c2b8d4f0145d7be03f31e97a53622: ending tablet copy session 9164d545640440dd848673f2a76a966d-255dbb2190684bbdb54920e474784a55 on tablet 255dbb2190684bbdb54920e474784a55 with peer 9164d545640440dd848673f2a76a966d
I20260923 22:16:03.393283 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.393194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:33837)
0923 22:16:03.393236 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.393263 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:16:03.789762 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.789558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:35487)
0923 22:16:03.789666 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.789707 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":45,"threads_started":1}
I20260923 22:16:03.789731 18412 raft_consensus.cc:1203] T 255dbb2190684bbdb54920e474784a55 P 9164d545640440dd848673f2a76a966d [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260923 22:16:03.790185 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.790066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:54261)
0923 22:16:03.790117 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.790153 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:16:03.939819 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:03.939612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:43023)
0923 22:16:03.939735 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:03.939771 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260923 22:16:04.790706 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.790459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:54839)
0923 22:16:04.790612 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.790659 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260923 22:16:04.790771 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.790582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:34529)
0923 22:16:04.790681 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.790714 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":45,"threads_started":1}
I20260923 22:16:04.940709 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:04.940396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:52035)
0923 22:16:04.940611 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:04.940657 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":106,"threads_started":1}
I20260923 22:16:05.791397 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.791161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:56017)
0923 22:16:05.791304 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.791349 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260923 22:16:05.791462 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.791200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:37017)
0923 22:16:05.791361 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.791395 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260923 22:16:05.941545 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:05.941237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:47345)
0923 22:16:05.941447 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:05.941491 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260923 22:16:06.792256 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.791978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:60913)
0923 22:16:06.792132 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.792204 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260923 22:16:06.792253 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.791974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:43323)
0923 22:16:06.792116 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.792169 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260923 22:16:06.792656 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.792520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:53207)
0923 22:16:06.792599 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.792631 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260923 22:16:06.942433 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:06.942163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:44497)
0923 22:16:06.942315 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:06.942385 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260923 22:16:07.793045 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.792802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:39421)
0923 22:16:07.792949 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.793000 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260923 22:16:07.793341 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.793075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:49469)
0923 22:16:07.793211 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.793248 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":48,"threads_started":1}
I20260923 22:16:07.943312 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:07.943000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:53433)
0923 22:16:07.943205 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:07.943243 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260923 22:16:08.793834 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.793562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:45977)
0923 22:16:08.793733 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:08.793779 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
I20260923 22:16:08.793998 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.793802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:43733)
0923 22:16:08.793913 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:08.793950 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":47,"threads_started":1}
I20260923 22:16:08.944135 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:08.943874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:55695)
0923 22:16:08.944033 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:08.944080 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260923 22:16:09.794528 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.794263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:49843)
0923 22:16:09.794434 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.794482 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260923 22:16:09.794543 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.794371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:34125)
0923 22:16:09.794471 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.794502 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":43,"threads_started":1}
I20260923 22:16:09.944983 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:09.944728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:55099)
0923 22:16:09.944882 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:09.944935 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260923 22:16:10.795192 18505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:10.794987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:47047)
0923 22:16:10.795106 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:10.795144 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260923 22:16:10.795222 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:10.794987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:33325)
0923 22:16:10.795149 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:10.795180 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260923 22:16:10.945640 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:10.945425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:46255)
0923 22:16:10.945555 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:10.945593 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260923 22:16:11.796000 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.795768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:46353)
0923 22:16:11.795911 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.795952 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260923 22:16:11.796008 18509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.795770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:34083)
0923 22:16:11.795928 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.795966 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260923 22:16:11.946321 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:11.946082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:51937)
0923 22:16:11.946230 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:11.946273 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260923 22:16:12.796808 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.796595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:56347)
0923 22:16:12.796722 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.796761 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260923 22:16:12.796811 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.796574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:40355)
0923 22:16:12.796721 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.796760 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260923 22:16:12.947058 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:12.946839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:33621)
0923 22:16:12.946964 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:12.947009 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260923 22:16:13.797555 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:13.797341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:48775)
0923 22:16:13.797469 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:13.797509 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260923 22:16:13.797555 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:13.797355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:58389)
0923 22:16:13.797474 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:13.797509 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260923 22:16:13.947889 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:13.947645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:38423)
0923 22:16:13.947793 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:13.947840 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260923 22:16:14.798422 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.798139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:33931)
0923 22:16:14.798312 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.798374 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
I20260923 22:16:14.798435 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.798138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:34713)
0923 22:16:14.798352 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.798393 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260923 22:16:14.948750 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:14.948509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:38151)
0923 22:16:14.948646 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:14.948685 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260923 22:16:15.799197 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.798967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.131:34089)
0923 22:16:15.799119 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.799152 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260923 22:16:15.799203 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.798967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.129:58405)
0923 22:16:15.799120 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.799160 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260923 22:16:15.949484 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:15.949219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40719 (local address 127.1.132.130:47861)
0923 22:16:15.949388 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:15.949436 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.190:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260923 22:16:16.465507 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18025
I20260923 22:16:16.469854 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18193
I20260923 22:16:16.473783 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18329
2026-09-23T22:16:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58251 ms)
[----------] 30 tests from SecurityITest (235061 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:16 dist-test-slave-l4q5 krb5kdc[18527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:16 dist-test-slave-l4q5 krb5kdc[18527](info): set up 2 sockets
Sep 23 22:16:16 dist-test-slave-l4q5 krb5kdc[18527](info): commencing operation
krb5kdc: starting...
W20260923 22:16:17.447814 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:17 dist-test-slave-l4q5 krb5kdc[18527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:17.568634 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40983
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:17.649006 18540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:17.649241 18540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:17.649276 18540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:17.650560 18540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:17.650611 18540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:17.650631 18540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:17.650646 18540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:17.650663 18540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:17.652302 18540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:17.652577 18540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:17.652801 18540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:17.658001 18545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:17.658041 18546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:17.658079 18540 server_base.cc:1034] running on GCE node
W20260923 22:16:17.658001 18548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:17.658665 18540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:17.659034 18540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:17.660225 18540 hybrid_clock.cc:648] HybridClock initialized: now 1790201777660225 us; error 61 us; skew 500 ppm
Sep 23 22:16:17 dist-test-slave-l4q5 krb5kdc[18527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:17.662775 18540 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:17.663229 18540 webserver.cc:460] Webserver started at http://127.1.132.190:37833/ using document root <none> and password file <none>
I20260923 22:16:17.663442 18540 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:17.663491 18540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:17.663612 18540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:17.664707 18540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "2673ab78b1274260aa292bbfd48a054b"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "016e8cfa22f5f665d831cf6f0a0fc56a"
server_key_iv: "eb28efe0f535987ff2cb812afcfb661f"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.665041 18540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "2673ab78b1274260aa292bbfd48a054b"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "016e8cfa22f5f665d831cf6f0a0fc56a"
server_key_iv: "eb28efe0f535987ff2cb812afcfb661f"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.666383 18540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:17.667296 18555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:17.667506 18540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:17.667613 18540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "2673ab78b1274260aa292bbfd48a054b"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "016e8cfa22f5f665d831cf6f0a0fc56a"
server_key_iv: "eb28efe0f535987ff2cb812afcfb661f"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.667682 18540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:17.677647 18540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:17.679778 18540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:17.679934 18540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:17.683672 18540 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40983
I20260923 22:16:17.683710 18607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40983 every 8 connection(s)
I20260923 22:16:17.684042 18540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:17.684953 18608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:17.687211 18608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: Bootstrap starting.
I20260923 22:16:17.687753 18608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:17.688014 18608 log.cc:824] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:17.688745 18608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: No bootstrap required, opened a new log
I20260923 22:16:17.690361 18608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } }
I20260923 22:16:17.690501 18608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:17.690522 18608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2673ab78b1274260aa292bbfd48a054b, State: Initialized, Role: FOLLOWER
I20260923 22:16:17.690665 18608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } }
I20260923 22:16:17.690730 18608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:17.690754 18608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:17.690800 18608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:17.691304 18608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } }
I20260923 22:16:17.691388 18608 leader_election.cc:302] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2673ab78b1274260aa292bbfd48a054b; no voters:
I20260923 22:16:17.691541 18608 leader_election.cc:288] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:17.691581 18611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:17.691735 18611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [term 1 LEADER]: Becoming Leader. State: Replica: 2673ab78b1274260aa292bbfd48a054b, State: Running, Role: LEADER
I20260923 22:16:17.691795 18608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:17.691824 18611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } }
I20260923 22:16:17.692272 18613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2673ab78b1274260aa292bbfd48a054b. Latest consensus state: current_term: 1 leader_uuid: "2673ab78b1274260aa292bbfd48a054b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } } }
I20260923 22:16:17.692373 18613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:17.692399 18612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2673ab78b1274260aa292bbfd48a054b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2673ab78b1274260aa292bbfd48a054b" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40983 } } }
I20260923 22:16:17.692467 18612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:17.692570 18620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:17.693092 18620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:17.693449 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18540
I20260923 22:16:17.693533 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:17.693686 1554 external_mini_cluster.cc:1428] Setting key 2b44a6d008dfdc4ff21be5452025ef40
I20260923 22:16:17.695848 18620 catalog_manager.cc:1347] Generated new cluster ID: 0c582acf42b341a9b4aa4432cd41d4b8
I20260923 22:16:17.695907 18620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:17 dist-test-slave-l4q5 krb5kdc[18527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:17.705593 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:17.694127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.1.132.190:40983)
0923 22:16:17.694296 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:17.694305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:17.694401 (+ 96us) server_negotiation.cc:407] Connection header received
0923 22:16:17.694561 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:17.694568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:17.694624 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:17.694764 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:17.695494 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.696981 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.698106 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.698751 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.698848 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:17.701997 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:17.702002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:17.702002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:17.702023 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:17.703936 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:17.704358 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:17.704361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:17.704362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:17.704476 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:17.704662 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:17.704663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:17.704663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:17.704978 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:17.705112 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:17.705217 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:16:17.705350 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260923 22:16:17.710109 18620 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:17.710922 18620 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:17.718819 18620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: Generated new TSK 0
I20260923 22:16:17.719059 18620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:16:17.747915 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:17.834177 18634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:17.834404 18634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:17.834430 18634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:17.835630 18634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:17.835670 18634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:17.835697 18634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:17.837227 18634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:17.837435 18634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:17.837633 18634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:17.842525 18639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:17.842530 18642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:17.842561 18640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:17.842739 18634 server_base.cc:1034] running on GCE node
I20260923 22:16:17.842964 18634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:17.843214 18634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:17.844355 18634 hybrid_clock.cc:648] HybridClock initialized: now 1790201777844342 us; error 29 us; skew 500 ppm
Sep 23 22:16:17 dist-test-slave-l4q5 krb5kdc[18527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:17.846851 18634 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:16:17.847325 18634 webserver.cc:460] Webserver started at http://127.1.132.129:33723/ using document root <none> and password file <none>
I20260923 22:16:17.847545 18634 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:17.847595 18634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:17.847723 18634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:17.848963 18634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "90166b5e57bf41f4b73914f000a9d711"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "789373d8b7dbe0178184b0899fd3be37"
server_key_iv: "d1250ed76ccf4c8c6b9c614840fc7fbd"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.849328 18634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "90166b5e57bf41f4b73914f000a9d711"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "789373d8b7dbe0178184b0899fd3be37"
server_key_iv: "d1250ed76ccf4c8c6b9c614840fc7fbd"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.850677 18634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:17.851432 18649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:17.851600 18634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:17.851697 18634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "90166b5e57bf41f4b73914f000a9d711"
format_stamp: "Formatted at 2026-09-23 22:16:17 on dist-test-slave-l4q5"
server_key: "789373d8b7dbe0178184b0899fd3be37"
server_key_iv: "d1250ed76ccf4c8c6b9c614840fc7fbd"
server_key_version: "encryptionkey@0"
I20260923 22:16:17.851761 18634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:17.880534 18634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:17.883066 18634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:17.883229 18634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:17.883467 18634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:17.883785 18634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:17.883819 18634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:17.883849 18634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:17.883865 18634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:17.890703 18634 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:38697
I20260923 22:16:17.890749 18762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:38697 every 8 connection(s)
I20260923 22:16:17.891103 18634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:17.892957 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18634
I20260923 22:16:17.893052 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:17.893210 1554 external_mini_cluster.cc:1428] Setting key 52b959f29df1ca3dabae9aa3b5f9941d
Sep 23 22:16:17 dist-test-slave-l4q5 krb5kdc[18527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201777, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
I20260923 22:16:17.901286 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:17.891858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:53321 (local address 127.1.132.190:40983)
0923 22:16:17.891906 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:17.891908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:17.892126 (+ 218us) server_negotiation.cc:407] Connection header received
0923 22:16:17.892636 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:17.892637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:17.892653 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:17.892714 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:17.893777 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.894276 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.895505 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.895717 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.895776 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:17.898337 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:17.898339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:17.898340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:17.898346 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:17.899792 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:17.900291 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:17.900292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:17.900292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:17.900346 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:17.900641 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:17.900642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:17.900642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:17.900746 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:17.900799 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:17.901145 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 22:16:17.901220 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:17.901548 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:17.891845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40983 (local address 127.1.132.129:53321)
0923 22:16:17.891993 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:17.892040 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:17.892555 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:17.892793 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:17.892797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:17.893470 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:17.893685 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:17.893690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.894365 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.894366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:17.895408 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:17.895410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:17.895803 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:17.895804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:17.896043 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:17.896692 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:17.896700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:17.898231 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:17.899881 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:17.899889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:17.899892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:17.900212 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:17.900434 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:17.900435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:17.900435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:17.900580 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:17.900810 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:17.900816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:17.901007 (+ 191us) client_negotiation.cc:769] Sending connection context
0923 22:16:17.901118 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 22:16:17.901291 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260923 22:16:17.902122 18763 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40983
I20260923 22:16:17.902211 18763 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:17.902400 18763 heartbeater.cc:511] Master 127.1.132.190:40983 requested a full tablet report, sending...
I20260923 22:16:17.903021 18572 ts_manager.cc:194] Registered new tserver with Master: 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697)
I20260923 22:16:17.903666 18572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:53321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:16:17.923260 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:18.008421 18768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.008642 18768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.008666 18768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.009896 18768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.009943 18768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.009972 18768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:18.011541 18768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:18.011787 18768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.012022 18768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.017483 18773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.017552 18768 server_base.cc:1034] running on GCE node
W20260923 22:16:18.017637 18776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.017500 18774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.017922 18768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.018183 18768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.019354 18768 hybrid_clock.cc:648] HybridClock initialized: now 1790201778019337 us; error 36 us; skew 500 ppm
Sep 23 22:16:18 dist-test-slave-l4q5 krb5kdc[18527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201778, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:18.022022 18768 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:16:18.022469 18768 webserver.cc:460] Webserver started at http://127.1.132.130:39789/ using document root <none> and password file <none>
I20260923 22:16:18.022693 18768 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.022756 18768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.022884 18768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.023933 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "3884573f532a4fd2b4d49ad4e7927d05"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "46c02d9c359c35166d7f3aea8112521e"
server_key_iv: "41ff7f817b747358108efebe622d259d"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.024307 18768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "3884573f532a4fd2b4d49ad4e7927d05"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "46c02d9c359c35166d7f3aea8112521e"
server_key_iv: "41ff7f817b747358108efebe622d259d"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.025660 18768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:18.026491 18783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.026739 18768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:18.026832 18768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "3884573f532a4fd2b4d49ad4e7927d05"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "46c02d9c359c35166d7f3aea8112521e"
server_key_iv: "41ff7f817b747358108efebe622d259d"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.026912 18768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.049768 18768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.052106 18768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.052294 18768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.052532 18768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:18.052834 18768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:18.052870 18768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.052899 18768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:18.052924 18768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.059749 18768 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:45077
I20260923 22:16:18.059832 18896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:45077 every 8 connection(s)
I20260923 22:16:18.060292 18768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:16:18 dist-test-slave-l4q5 krb5kdc[18527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201778, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:18.068602 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18768
I20260923 22:16:18.068703 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:18.068898 1554 external_mini_cluster.cc:1428] Setting key 6cea07b61fb61f3c475510c0ab387834
I20260923 22:16:18.070878 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.061216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:38819 (local address 127.1.132.190:40983)
0923 22:16:18.061290 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.061292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.061462 (+ 170us) server_negotiation.cc:407] Connection header received
0923 22:16:18.061988 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.061990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.062002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.062065 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.063168 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.063689 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.064868 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.065092 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.065163 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.067521 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.067523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.067523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:18.067531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.069287 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.069790 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.069792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.069792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.069883 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.070130 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.070131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.070132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.070290 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.070358 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.070684 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.070786 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:16:18.071131 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.061158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40983 (local address 127.1.132.130:38819)
0923 22:16:18.061331 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.061376 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.061850 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.062120 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.062124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.062822 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:18.063037 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.063042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.063804 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.063805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.064750 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.064753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.065212 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.065213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.065338 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.065882 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:18.065890 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:18.067398 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:18.069366 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:18.069403 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:18.069410 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:18.069725 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:18.069957 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:18.069958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:18.069958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:18.070074 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:18.070389 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:18.070397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:18.070551 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.070653 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.070827 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260923 22:16:18.071707 18897 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40983
I20260923 22:16:18.071806 18897 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:18.071990 18897 heartbeater.cc:511] Master 127.1.132.190:40983 requested a full tablet report, sending...
I20260923 22:16:18.072395 18572 ts_manager.cc:194] Registered new tserver with Master: 3884573f532a4fd2b4d49ad4e7927d05 (127.1.132.130:45077)
I20260923 22:16:18.072875 18572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:38819
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:16:18.100503 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:18.190974 18902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.191231 18902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.191257 18902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.192723 18902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.192786 18902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.192829 18902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:18.194949 18902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:40983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:18.195300 18902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.195631 18902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.203043 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.203047 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.203043 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.203450 18902 server_base.cc:1034] running on GCE node
I20260923 22:16:18.203676 18902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.204035 18902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.205252 18902 hybrid_clock.cc:648] HybridClock initialized: now 1790201778205235 us; error 32 us; skew 500 ppm
Sep 23 22:16:18 dist-test-slave-l4q5 krb5kdc[18527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201778, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:18.208279 18902 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:16:18.208897 18902 webserver.cc:460] Webserver started at http://127.1.132.131:44953/ using document root <none> and password file <none>
I20260923 22:16:18.209141 18902 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.209193 18902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.209311 18902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.210377 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "7e9d26cee0de4ebb889f594d2e10bb6d"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "c6743c8ead783c7fc650cf504464eb3a"
server_key_iv: "c97ceb24589d31258855a6ffc81560aa"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.210716 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "7e9d26cee0de4ebb889f594d2e10bb6d"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "c6743c8ead783c7fc650cf504464eb3a"
server_key_iv: "c97ceb24589d31258855a6ffc81560aa"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.212128 18902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:18.213078 18917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.213299 18902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:18.213382 18902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "7e9d26cee0de4ebb889f594d2e10bb6d"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "c6743c8ead783c7fc650cf504464eb3a"
server_key_iv: "c97ceb24589d31258855a6ffc81560aa"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.213466 18902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.232024 18902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.234193 18902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.234318 18902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.234562 18902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:18.234875 18902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:18.234911 18902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.234941 18902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:18.234957 18902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.240643 18902 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:35999
I20260923 22:16:18.240729 19030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:35999 every 8 connection(s)
I20260923 22:16:18.241026 18902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:18.246165 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 18902
I20260923 22:16:18.246275 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:18.246466 1554 external_mini_cluster.cc:1428] Setting key ec5e16a487521655ec7ae57a6e4ec110
Sep 23 22:16:18 dist-test-slave-l4q5 krb5kdc[18527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201778, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:18.251091 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.241911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:54889 (local address 127.1.132.190:40983)
0923 22:16:18.241963 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.241964 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.242179 (+ 215us) server_negotiation.cc:407] Connection header received
0923 22:16:18.242732 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.242734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.242749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.242804 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.243804 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.244346 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.245380 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.245608 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.245668 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.248173 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.248175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.248175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:18.248213 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.249706 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.250150 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.250151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.250152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.250210 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.250440 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.250441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.250441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.250572 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.250629 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.250908 (+ 279us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.250993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:18.251300 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.241844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:40983 (local address 127.1.132.131:54889)
0923 22:16:18.242048 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.242087 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.242614 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.242896 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.242901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.243488 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:18.243694 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.243699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.244432 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.244433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.245275 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.245277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.245689 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.245690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.245818 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.246345 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:18.246364 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:18.248071 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:18.249804 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:18.249815 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:18.249818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:18.250078 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:18.250266 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:18.250267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:18.250267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:18.250383 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:18.250628 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:18.250636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:18.250785 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.250878 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.251075 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260923 22:16:18.251722 19031 heartbeater.cc:347] Connected to a master server at 127.1.132.190:40983
I20260923 22:16:18.251819 19031 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:18.251973 19031 heartbeater.cc:511] Master 127.1.132.190:40983 requested a full tablet report, sending...
I20260923 22:16:18.252370 18572 ts_manager.cc:194] Registered new tserver with Master: 7e9d26cee0de4ebb889f594d2e10bb6d (127.1.132.131:35999)
I20260923 22:16:18.252725 18572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:54889
I20260923 22:16:18.259258 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:18.267725 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.261292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.1.132.190:40983)
0923 22:16:18.261348 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.261350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.261472 (+ 122us) server_negotiation.cc:407] Connection header received
0923 22:16:18.261532 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.261533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.261544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.261599 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.262410 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.262927 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.263768 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.264076 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.264147 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.265405 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.265409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.265409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:18.265419 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.266715 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.266991 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.266992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.266992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.267040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:18.267226 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:18.267228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:18.267228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:18.267415 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.267509 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.267592 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.267658 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:16:18.269524 18572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:18.271190 18572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:18.277897 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.1.132.130:45077)
0923 22:16:18.274145 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.274158 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.274168 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:18.274208 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.274210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.274226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.274296 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.274522 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275646 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.276757 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.277418 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277497 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.277558 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.277766 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.277813 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":69,"threads_started":1}
I20260923 22:16:18.277875 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:45077 (local address 127.0.0.1:48988)
0923 22:16:18.274070 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.274080 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.274133 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.274376 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.274377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.274379 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.274445 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.274446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275743 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.275744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.276676 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.276679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.277543 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.277624 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.277638 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.277722 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.277783 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":350,"threads_started":1}
I20260923 22:16:18.278373 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:38697 (local address 127.0.0.1:49504)
0923 22:16:18.273827 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.273872 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.273940 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.274208 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.274211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.274213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.274279 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.274281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275669 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.275670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.276953 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.276954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.277867 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.278175 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.278181 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.278252 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.278299 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":5,"thread_start_us":63,"threads_started":1}
I20260923 22:16:18.278671 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.1.132.129:38697)
0923 22:16:18.273971 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.273975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.273985 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:18.274024 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.274026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.274041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.274110 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.274351 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275566 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277032 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.277720 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277783 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.277831 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.278542 (+ 711us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.278594 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260923 22:16:18.278661 18831 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 (DEFAULT_TABLE table=test-table [id=bba3c3e5ca2746aebb320211cbf28c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:18.278996 19043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:35999 (local address 127.0.0.1:55932)
0923 22:16:18.274201 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.274209 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.274257 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.274561 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.274562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.274564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.274610 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.274612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275516 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.275518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.276930 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.276934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.278663 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.278664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.278822 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.278828 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.278876 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.278924 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":11,"spinlock_wait_cycles":896,"thread_start_us":112,"threads_started":1}
I20260923 22:16:18.279063 18831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe897d54d04bc59d7717c5c748fa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:18.279310 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.273953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.1.132.131:35999)
0923 22:16:18.274114 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.274118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.274267 (+ 149us) server_negotiation.cc:407] Connection header received
0923 22:16:18.274325 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.274327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.274343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.274453 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.274689 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.275408 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.277657 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.278515 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.278577 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.278630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.279085 (+ 455us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.279137 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260923 22:16:18.279976 18697 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 (DEFAULT_TABLE table=test-table [id=bba3c3e5ca2746aebb320211cbf28c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:18.280328 18697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe897d54d04bc59d7717c5c748fa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:18.280292 18965 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 (DEFAULT_TABLE table=test-table [id=bba3c3e5ca2746aebb320211cbf28c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:18.280660 18965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe897d54d04bc59d7717c5c748fa24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:18.283308 19048 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Bootstrap starting.
I20260923 22:16:18.283784 19049 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Bootstrap starting.
I20260923 22:16:18.284011 19048 tablet_bootstrap.cc:654] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:18.284484 19049 tablet_bootstrap.cc:654] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:18.284611 19050 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Bootstrap starting.
I20260923 22:16:18.284611 19048 log.cc:824] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:18.285187 19049 log.cc:824] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:18.285358 19050 tablet_bootstrap.cc:654] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:18.285600 19048 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: No bootstrap required, opened a new log
I20260923 22:16:18.285686 19048 ts_tablet_manager.cc:1397] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:18.285913 19050 log.cc:824] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:18.285984 19049 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: No bootstrap required, opened a new log
I20260923 22:16:18.286031 19049 ts_tablet_manager.cc:1397] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:18.286672 19050 tablet_bootstrap.cc:492] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: No bootstrap required, opened a new log
I20260923 22:16:18.286729 19050 ts_tablet_manager.cc:1397] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:18.287532 19049 raft_consensus.cc:348] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.287721 19049 raft_consensus.cc:374] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:18.287761 19049 raft_consensus.cc:729] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90166b5e57bf41f4b73914f000a9d711, State: Initialized, Role: FOLLOWER
I20260923 22:16:18.287693 19048 raft_consensus.cc:348] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.287858 19048 raft_consensus.cc:374] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:18.287891 19048 raft_consensus.cc:729] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884573f532a4fd2b4d49ad4e7927d05, State: Initialized, Role: FOLLOWER
I20260923 22:16:18.287873 19049 consensus_queue.cc:260] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.288007 19048 consensus_queue.cc:260] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.288157 19049 ts_tablet_manager.cc:1428] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:18.288157 18763 heartbeater.cc:503] Master 127.1.132.190:40983 was elected leader, sending a full tablet report...
I20260923 22:16:18.288300 19048 ts_tablet_manager.cc:1428] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:18.288412 18897 heartbeater.cc:503] Master 127.1.132.190:40983 was elected leader, sending a full tablet report...
I20260923 22:16:18.288719 19050 raft_consensus.cc:348] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.288887 19050 raft_consensus.cc:374] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:18.288937 19050 raft_consensus.cc:729] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e9d26cee0de4ebb889f594d2e10bb6d, State: Initialized, Role: FOLLOWER
I20260923 22:16:18.289085 19050 consensus_queue.cc:260] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.289382 19050 ts_tablet_manager.cc:1428] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:18.289636 19031 heartbeater.cc:503] Master 127.1.132.190:40983 was elected leader, sending a full tablet report...
W20260923 22:16:18.310954 18898 tablet.cc:2369] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:18.392069 18764 tablet.cc:2369] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:18.393554 19055 raft_consensus.cc:482] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:18.393699 19055 raft_consensus.cc:504] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.393755 19056 raft_consensus.cc:482] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:18.393852 19056 raft_consensus.cc:504] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.394135 19056 leader_election.cc:288] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697), 3884573f532a4fd2b4d49ad4e7927d05 (127.1.132.130:45077)
I20260923 22:16:18.395615 19055 leader_election.cc:288] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697), 7e9d26cee0de4ebb889f594d2e10bb6d (127.1.132.131:35999)
I20260923 22:16:18.397763 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:35999 (local address 127.1.132.130:37155)
0923 22:16:18.394354 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.394366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.394409 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.394625 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.394626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.394628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.394700 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.394702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395594 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.395595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.396823 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.396826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.397512 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.397513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.397598 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.397601 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.397641 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.397684 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:16:18.397863 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:38697 (local address 127.1.132.131:51661)
0923 22:16:18.394369 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.394377 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.394413 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.394737 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.394737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.394739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.394809 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.394810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395627 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.395627 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.396891 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.396894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.397633 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.397634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.397708 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.397711 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.397752 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.397791 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":121}
I20260923 22:16:18.398036 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:37155 (local address 127.1.132.131:35999)
0923 22:16:18.394477 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.394479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.394487 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:18.394508 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.394509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.394517 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.394573 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.394769 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395513 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.396929 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.397404 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.397473 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.397482 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.397933 (+ 451us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.397971 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260923 22:16:18.398056 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:51661 (local address 127.1.132.129:38697)
0923 22:16:18.394641 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.394643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.394649 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:16:18.394671 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.394672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.394681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.394730 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.394863 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395545 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.396969 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.397528 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.397594 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.397602 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.397956 (+ 354us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.397992 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260923 22:16:18.398303 18985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "3884573f532a4fd2b4d49ad4e7927d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" is_pre_election: true
I20260923 22:16:18.398427 18985 raft_consensus.cc:2454] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884573f532a4fd2b4d49ad4e7927d05 in term 0.
I20260923 22:16:18.398388 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90166b5e57bf41f4b73914f000a9d711" is_pre_election: true
I20260923 22:16:18.398545 18717 raft_consensus.cc:2454] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e9d26cee0de4ebb889f594d2e10bb6d in term 0.
I20260923 22:16:18.398722 19057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:38697 (local address 127.1.132.130:44885)
0923 22:16:18.395746 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.395755 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.395806 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.395936 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.395937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.395939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.396039 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.396041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.396694 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.396694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.397744 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.397747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.398483 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.398484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.398551 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.398555 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.398606 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.398648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1434,"mutex_wait_us":36,"thread_start_us":156,"threads_started":1}
I20260923 22:16:18.398777 18919 leader_election.cc:302] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e9d26cee0de4ebb889f594d2e10bb6d, 90166b5e57bf41f4b73914f000a9d711; no voters:
I20260923 22:16:18.398744 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:44885 (local address 127.1.132.129:38697)
0923 22:16:18.394786 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.394788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.395824 (+ 1036us) server_negotiation.cc:407] Connection header received
0923 22:16:18.395852 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.395853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.395862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.395922 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.396104 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.396619 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.397821 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.398371 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.398440 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.398449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.398643 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.398681 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":61,"threads_started":1}
I20260923 22:16:18.398998 18786 leader_election.cc:302] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884573f532a4fd2b4d49ad4e7927d05, 7e9d26cee0de4ebb889f594d2e10bb6d; no voters:
I20260923 22:16:18.399133 19055 raft_consensus.cc:2781] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:18.399195 19055 raft_consensus.cc:482] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:18.399220 19055 raft_consensus.cc:3037] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:18.399221 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "3884573f532a4fd2b4d49ad4e7927d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90166b5e57bf41f4b73914f000a9d711" is_pre_election: true
I20260923 22:16:18.399291 18717 raft_consensus.cc:2454] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884573f532a4fd2b4d49ad4e7927d05 in term 0.
I20260923 22:16:18.399439 19056 raft_consensus.cc:2781] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:18.399502 19056 raft_consensus.cc:482] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:18.399523 19056 raft_consensus.cc:3037] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:18.399925 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:47459 (local address 127.1.132.130:45077)
0923 22:16:18.394661 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.394662 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.394671 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:18.394791 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.394792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.394800 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.394826 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:18.395002 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395462 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.399075 (+ 3613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.399615 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.399685 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.399694 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.399834 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.399870 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:18.399946 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.394462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:45077 (local address 127.1.132.131:47459)
0923 22:16:18.394604 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.394614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.394715 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.394888 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.394888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.394890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:18.394945 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.394946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.395542 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.395543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.398995 (+ 3452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.398998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.399696 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.399697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.399771 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.399776 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.399821 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.399861 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260923 22:16:18.400167 19055 raft_consensus.cc:504] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.400345 19055 leader_election.cc:288] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [CANDIDATE]: Term 1 election: Requested vote from peers 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697), 7e9d26cee0de4ebb889f594d2e10bb6d (127.1.132.131:35999)
I20260923 22:16:18.400310 19056 raft_consensus.cc:504] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.400430 19056 leader_election.cc:288] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [CANDIDATE]: Term 1 election: Requested vote from peers 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697), 3884573f532a4fd2b4d49ad4e7927d05 (127.1.132.130:45077)
I20260923 22:16:18.400497 18851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884573f532a4fd2b4d49ad4e7927d05" is_pre_election: true
I20260923 22:16:18.400600 18851 raft_consensus.cc:2379] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e9d26cee0de4ebb889f594d2e10bb6d in current term 1: Already voted for candidate 3884573f532a4fd2b4d49ad4e7927d05 in this term.
I20260923 22:16:18.400578 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "3884573f532a4fd2b4d49ad4e7927d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90166b5e57bf41f4b73914f000a9d711"
I20260923 22:16:18.400621 18985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "3884573f532a4fd2b4d49ad4e7927d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d"
I20260923 22:16:18.400640 18717 raft_consensus.cc:3037] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:18.400714 18985 raft_consensus.cc:2379] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3884573f532a4fd2b4d49ad4e7927d05 in current term 1: Already voted for candidate 7e9d26cee0de4ebb889f594d2e10bb6d in this term.
I20260923 22:16:18.400760 18716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90166b5e57bf41f4b73914f000a9d711"
I20260923 22:16:18.401427 18717 raft_consensus.cc:2454] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884573f532a4fd2b4d49ad4e7927d05 in term 1.
I20260923 22:16:18.401590 18785 leader_election.cc:302] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3884573f532a4fd2b4d49ad4e7927d05, 90166b5e57bf41f4b73914f000a9d711; no voters: 7e9d26cee0de4ebb889f594d2e10bb6d
I20260923 22:16:18.401628 18851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe897d54d04bc59d7717c5c748fa24" candidate_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884573f532a4fd2b4d49ad4e7927d05"
I20260923 22:16:18.401688 19055 raft_consensus.cc:2781] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:18.401829 19055 raft_consensus.cc:686] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 LEADER]: Becoming Leader. State: Replica: 3884573f532a4fd2b4d49ad4e7927d05, State: Running, Role: LEADER
I20260923 22:16:18.401947 19055 consensus_queue.cc:237] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } }
I20260923 22:16:18.402480 18921 leader_election.cc:302] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e9d26cee0de4ebb889f594d2e10bb6d; no voters: 3884573f532a4fd2b4d49ad4e7927d05, 90166b5e57bf41f4b73914f000a9d711
I20260923 22:16:18.402562 19056 raft_consensus.cc:2726] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 22:16:18.402730 18572 catalog_manager.cc:5495] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3884573f532a4fd2b4d49ad4e7927d05 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "3884573f532a4fd2b4d49ad4e7927d05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884573f532a4fd2b4d49ad4e7927d05" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:18.426070 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.423895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.1.132.130:45077)
0923 22:16:18.423946 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.423948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.423957 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:18.424000 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.424000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.424027 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.424088 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:18.424242 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.424728 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.425431 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.425619 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.425673 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.425777 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:18.425891 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:18.425953 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.425971 (+ 18us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.425988 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:16:18.427896 18985 raft_consensus.cc:1261] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Refusing update from remote peer 3884573f532a4fd2b4d49ad4e7927d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:18.427917 18717 raft_consensus.cc:1261] T 6dbe897d54d04bc59d7717c5c748fa24 P 90166b5e57bf41f4b73914f000a9d711 [term 1 FOLLOWER]: Refusing update from remote peer 3884573f532a4fd2b4d49ad4e7927d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:18.428171 19060 consensus_queue.cc:1035] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e9d26cee0de4ebb889f594d2e10bb6d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:18.428330 19055 consensus_queue.cc:1035] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90166b5e57bf41f4b73914f000a9d711" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:18.430876 19063 mvcc.cc:205] Tried to move back new op lower bound from 7332666484437929984 to 7332666484334870528. Current Snapshot: MvccSnapshot[applied={T|T < 7332666484437929984}]
I20260923 22:16:18.430995 19065 mvcc.cc:205] Tried to move back new op lower bound from 7332666484437929984 to 7332666484334870528. Current Snapshot: MvccSnapshot[applied={T|T < 7332666484437929984}]
I20260923 22:16:18.431205 19064 mvcc.cc:205] Tried to move back new op lower bound from 7332666484437929984 to 7332666484334870528. Current Snapshot: MvccSnapshot[applied={T|T < 7332666484437929984}]
I20260923 22:16:18.433431 18572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:18.433521 18572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:18.434317 18572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2673ab78b1274260aa292bbfd48a054b: Sending DeleteTablet for 3 replicas of tablet 6dbe897d54d04bc59d7717c5c748fa24
I20260923 22:16:18.434680 18697 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:18 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:49504
I20260923 22:16:18.434725 18965 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:18 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:55932
I20260923 22:16:18.434752 18831 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dbe897d54d04bc59d7717c5c748fa24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:18 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:48988
I20260923 22:16:18.435041 19075 tablet_replica.cc:330] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: stopping tablet replica
I20260923 22:16:18.435145 19075 raft_consensus.cc:2229] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:18.435266 19075 raft_consensus.cc:2258] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:18.435300 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18634
I20260923 22:16:18.436296 19075 ts_tablet_manager.cc:1905] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:18.437841 19074 tablet_replica.cc:330] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: stopping tablet replica
I20260923 22:16:18.438082 19074 raft_consensus.cc:2229] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:18.438269 19074 raft_consensus.cc:2258] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:18.438265 19075 ts_tablet_manager.cc:1918] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:18.438330 19075 log.cc:1196] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/6dbe897d54d04bc59d7717c5c748fa24
I20260923 22:16:18.438620 19075 ts_tablet_manager.cc:1939] T 6dbe897d54d04bc59d7717c5c748fa24 P 3884573f532a4fd2b4d49ad4e7927d05: Deleting consensus metadata
I20260923 22:16:18.439042 18559 catalog_manager.cc:4841] TS 3884573f532a4fd2b4d49ad4e7927d05 (127.1.132.130:45077): tablet 6dbe897d54d04bc59d7717c5c748fa24 (table test-table [id=bba3c3e5ca2746aebb320211cbf28c9e]) successfully deleted
I20260923 22:16:18.440889 19074 ts_tablet_manager.cc:1905] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:16:18.441785 18557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:38697 (error 108)
W20260923 22:16:18.442341 19046 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:18.442198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:38697 (local address 127.0.0.1:49518)
0923 22:16:18.442251 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.442294 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:38697: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:16:18.442463 18557 catalog_manager.cc:4584] TS 90166b5e57bf41f4b73914f000a9d711 (127.1.132.129:38697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dbe897d54d04bc59d7717c5c748fa24: Network error: Client connection negotiation failed: client connection to 127.1.132.129:38697: connect: Connection reset by peer (error 104)
I20260923 22:16:18.442667 19074 ts_tablet_manager.cc:1918] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:18.442744 19074 log.cc:1196] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/6dbe897d54d04bc59d7717c5c748fa24
I20260923 22:16:18.442750 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18768
I20260923 22:16:18.442991 19074 ts_tablet_manager.cc:1939] T 6dbe897d54d04bc59d7717c5c748fa24 P 7e9d26cee0de4ebb889f594d2e10bb6d: Deleting consensus metadata
I20260923 22:16:18.443660 18558 catalog_manager.cc:4841] TS 7e9d26cee0de4ebb889f594d2e10bb6d (127.1.132.131:35999): tablet 6dbe897d54d04bc59d7717c5c748fa24 (table test-table [id=bba3c3e5ca2746aebb320211cbf28c9e]) successfully deleted
I20260923 22:16:18.447777 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18902
I20260923 22:16:18.451817 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 18540
2026-09-23T22:16:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1976 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T22:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:18Z Disabled control of system clock
I20260923 22:16:18.478951 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34699
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:34699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:18.562173 19080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.562430 19080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.562453 19080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.563632 19080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:18.563666 19080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.563678 19080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.563687 19080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:18.563699 19080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:18.565042 19080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:34699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:34699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:18.565253 19080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.565512 19080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.570638 19085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.570648 19088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.570648 19086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.570957 19080 server_base.cc:1034] running on GCE node
I20260923 22:16:18.571305 19080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.571676 19080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.572858 19080 hybrid_clock.cc:648] HybridClock initialized: now 1790201778572839 us; error 37 us; skew 500 ppm
I20260923 22:16:18.574534 19080 webserver.cc:460] Webserver started at http://127.1.132.190:41829/ using document root <none> and password file <none>
I20260923 22:16:18.574813 19080 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.574863 19080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.574985 19080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.576113 19080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "765a3522f8ad4d779a599eb6e4cd7183"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "b6b049df4612ce6e3ff6ba4b66fe323a"
server_key_iv: "1312b02e3a6b13cb9a5c9dc108ea565a"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.576527 19080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "765a3522f8ad4d779a599eb6e4cd7183"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "b6b049df4612ce6e3ff6ba4b66fe323a"
server_key_iv: "1312b02e3a6b13cb9a5c9dc108ea565a"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.577960 19080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:18.578791 19094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.578995 19080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:18.579078 19080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "765a3522f8ad4d779a599eb6e4cd7183"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "b6b049df4612ce6e3ff6ba4b66fe323a"
server_key_iv: "1312b02e3a6b13cb9a5c9dc108ea565a"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.579162 19080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.599768 19080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.600114 19080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.600283 19080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.603889 19080 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:34699
I20260923 22:16:18.603943 19146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:34699 every 8 connection(s)
I20260923 22:16:18.604285 19080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:18.605288 19147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:18.607563 19147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: Bootstrap starting.
I20260923 22:16:18.608122 19147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:18.608435 19147 log.cc:824] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:18.609184 19147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: No bootstrap required, opened a new log
I20260923 22:16:18.610778 19147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } }
I20260923 22:16:18.610929 19147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:18.610952 19147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765a3522f8ad4d779a599eb6e4cd7183, State: Initialized, Role: FOLLOWER
I20260923 22:16:18.611101 19147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } }
I20260923 22:16:18.611164 19147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:18.611184 19147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:18.611212 19147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:18.611744 19147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } }
I20260923 22:16:18.611836 19147 leader_election.cc:302] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765a3522f8ad4d779a599eb6e4cd7183; no voters:
I20260923 22:16:18.612004 19147 leader_election.cc:288] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:18.612066 19150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:18.612257 19150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [term 1 LEADER]: Becoming Leader. State: Replica: 765a3522f8ad4d779a599eb6e4cd7183, State: Running, Role: LEADER
I20260923 22:16:18.612277 19147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:18.612375 19150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } }
I20260923 22:16:18.612798 19151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "765a3522f8ad4d779a599eb6e4cd7183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } } }
I20260923 22:16:18.612886 19151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:18.613058 19152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765a3522f8ad4d779a599eb6e4cd7183. Latest consensus state: current_term: 1 leader_uuid: "765a3522f8ad4d779a599eb6e4cd7183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765a3522f8ad4d779a599eb6e4cd7183" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 34699 } } }
I20260923 22:16:18.613098 19159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:18.613122 19152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:18.613595 19159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:18.613911 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19080
I20260923 22:16:18.614004 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:18.614145 1554 external_mini_cluster.cc:1428] Setting key 9c9a63f56c38e44415dc90614cd41810
I20260923 22:16:18.615185 19159 catalog_manager.cc:1347] Generated new cluster ID: 75d39f1ad16c4a0fa12a8caa1642978e
I20260923 22:16:18.615233 19159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:18.618913 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.614586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.1.132.190:34699)
0923 22:16:18.614832 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.614841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.614882 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:16:18.614981 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.614987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.615038 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.615209 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.615410 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.616344 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.617112 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.617536 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.617646 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.618267 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.618271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.618271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:18.618444 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.618477 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:18.618488 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.618544 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.618614 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.618694 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":62,"threads_started":1}
I20260923 22:16:18.628131 19159 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:18.628684 19159 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:18.638371 19159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: Generated new TSK 0
I20260923 22:16:18.638585 19159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:18.644007 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--builtin_ntp_servers=127.1.132.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:18.749676 19171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.749908 19171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.749933 19171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.751246 19171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.751302 19171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.751329 19171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:18.752898 19171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:18.753165 19171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.753471 19171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.759994 19179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.759994 19177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.760353 19176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.761337 19171 server_base.cc:1034] running on GCE node
I20260923 22:16:18.761649 19171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.762245 19171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.763449 19171 hybrid_clock.cc:648] HybridClock initialized: now 1790201778763520 us; error 130 us; skew 500 ppm
I20260923 22:16:18.765228 19171 webserver.cc:460] Webserver started at http://127.1.132.129:45545/ using document root <none> and password file <none>
I20260923 22:16:18.765519 19171 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.765589 19171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.765733 19171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.767140 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "3dbe7eebdd444853a22d494ff4a2438e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "5767f60f52424d6d09cdc2deafaafb3e"
server_key_iv: "79f191cffb5fb3ff9b775fcba176bf82"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.767534 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "3dbe7eebdd444853a22d494ff4a2438e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "5767f60f52424d6d09cdc2deafaafb3e"
server_key_iv: "79f191cffb5fb3ff9b775fcba176bf82"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.769208 19171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 22:16:18.770124 19185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.770401 19171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:18.770500 19171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "3dbe7eebdd444853a22d494ff4a2438e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "5767f60f52424d6d09cdc2deafaafb3e"
server_key_iv: "79f191cffb5fb3ff9b775fcba176bf82"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.770583 19171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.790460 19171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.790796 19171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.790930 19171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.791177 19171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:18.791695 19171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:18.791742 19171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.791776 19171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:18.791793 19171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.800591 19171 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:39409
I20260923 22:16:18.800661 19298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:39409 every 8 connection(s)
I20260923 22:16:18.801019 19171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:18.807374 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.801913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:56233 (local address 127.1.132.190:34699)
0923 22:16:18.802086 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.802089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.802363 (+ 274us) server_negotiation.cc:407] Connection header received
0923 22:16:18.803159 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.803161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.803174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.803224 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.804000 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.804687 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.805795 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.806006 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.806067 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.807030 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.807031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.807031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:18.807037 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.807053 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:18.807058 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.807106 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.807271 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.807303 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 22:16:18.807521 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.801770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34699 (local address 127.1.132.129:56233)
0923 22:16:18.802085 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.802124 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.803009 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.803331 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.803336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.803346 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:18.803868 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.803873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.804815 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.804816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.805681 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.805684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.806113 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.806114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.806247 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.806813 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:18.806814 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:18.806845 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:18.806880 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:18.807138 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:18.807145 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:18.807148 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.807211 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.807314 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":141,"threads_started":1}
I20260923 22:16:18.809859 19299 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34699
I20260923 22:16:18.810010 19299 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:18.810212 19299 heartbeater.cc:511] Master 127.1.132.190:34699 requested a full tablet report, sending...
I20260923 22:16:18.810554 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19171
I20260923 22:16:18.810653 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:18.810876 1554 external_mini_cluster.cc:1428] Setting key 7d4ddc257868674723e7e8f48580d114
I20260923 22:16:18.811228 19111 ts_manager.cc:194] Registered new tserver with Master: 3dbe7eebdd444853a22d494ff4a2438e (127.1.132.129:39409)
I20260923 22:16:18.812028 19111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.129:56233
I20260923 22:16:18.812263 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--builtin_ntp_servers=127.1.132.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:18.894958 19302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:18.895227 19302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:18.895251 19302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:18.896524 19302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:18.896574 19302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:18.896601 19302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:18.898041 19302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:18.898264 19302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:18.898536 19302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:18.903898 19307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:18.903951 19308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.904099 19302 server_base.cc:1034] running on GCE node
W20260923 22:16:18.903946 19310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:18.904505 19302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:18.904721 19302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:18.905864 19302 hybrid_clock.cc:648] HybridClock initialized: now 1790201778905851 us; error 28 us; skew 500 ppm
I20260923 22:16:18.907042 19302 webserver.cc:460] Webserver started at http://127.1.132.130:38387/ using document root <none> and password file <none>
I20260923 22:16:18.907262 19302 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:18.907335 19302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:18.907464 19302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:18.908553 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "bcd59929d86c44259705223ef9bb7f4e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "f92d5e3a7e48c00dc7ed3fef6badd65c"
server_key_iv: "f91c380b38c82ba3158f27f5e41ab835"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.908859 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "bcd59929d86c44259705223ef9bb7f4e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "f92d5e3a7e48c00dc7ed3fef6badd65c"
server_key_iv: "f91c380b38c82ba3158f27f5e41ab835"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.910079 19302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:18.910845 19316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.911011 19302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:18.911087 19302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "bcd59929d86c44259705223ef9bb7f4e"
format_stamp: "Formatted at 2026-09-23 22:16:18 on dist-test-slave-l4q5"
server_key: "f92d5e3a7e48c00dc7ed3fef6badd65c"
server_key_iv: "f91c380b38c82ba3158f27f5e41ab835"
server_key_version: "encryptionkey@0"
I20260923 22:16:18.911142 19302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:18.924472 19302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:18.924758 19302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:18.924863 19302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:18.925045 19302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:18.925357 19302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:18.925393 19302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.925415 19302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:18.925438 19302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:18.930850 19302 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:45595
I20260923 22:16:18.930936 19429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:45595 every 8 connection(s)
I20260923 22:16:18.931228 19302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:18.936408 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.932010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:47931 (local address 127.1.132.190:34699)
0923 22:16:18.932100 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:18.932102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:18.932334 (+ 232us) server_negotiation.cc:407] Connection header received
0923 22:16:18.932815 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:18.932816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:18.932826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:18.932876 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:18.933495 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.933990 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.935036 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.935232 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.935291 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.936165 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:18.936166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:18.936166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:18.936170 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:18.936209 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:18.936213 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:18.936256 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:18.936323 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:16:18.936344 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 22:16:18.936635 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:18.931966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34699 (local address 127.1.132.130:47931)
0923 22:16:18.932172 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:18.932247 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:18.932730 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:18.932932 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:18.932936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:18.932944 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:18.933431 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.933435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.934085 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.934086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.934941 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:18.934944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:18.935331 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:18.935332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:18.935459 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:18.936036 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:18.936036 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:18.936067 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:18.936094 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:18.936258 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:18.936262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:18.936264 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:18.936321 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:16:18.936412 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260923 22:16:18.937182 19430 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34699
I20260923 22:16:18.937278 19430 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:18.937476 19430 heartbeater.cc:511] Master 127.1.132.190:34699 requested a full tablet report, sending...
I20260923 22:16:18.937846 19111 ts_manager.cc:194] Registered new tserver with Master: bcd59929d86c44259705223ef9bb7f4e (127.1.132.130:45595)
I20260923 22:16:18.938035 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19302
I20260923 22:16:18.938117 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:18.938323 19111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.130:47931
I20260923 22:16:18.938347 1554 external_mini_cluster.cc:1428] Setting key d30774105462ea27edc715c54187fc76
I20260923 22:16:18.939567 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--builtin_ntp_servers=127.1.132.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:19.021727 19433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:19.021958 19433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:19.021982 19433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:19.023146 19433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:19.023187 19433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:19.023212 19433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:19.024581 19433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:34699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:19.024786 19433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:19.025053 19433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:19.029830 19439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:19.029887 19438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:19.029825 19441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:19.030359 19433 server_base.cc:1034] running on GCE node
I20260923 22:16:19.030560 19433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:19.030766 19433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:19.031901 19433 hybrid_clock.cc:648] HybridClock initialized: now 1790201779031892 us; error 28 us; skew 500 ppm
I20260923 22:16:19.033176 19433 webserver.cc:460] Webserver started at http://127.1.132.131:39435/ using document root <none> and password file <none>
I20260923 22:16:19.033421 19433 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:19.033469 19433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:19.033584 19433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:19.034641 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "83b6c6049e8542eca54337e30047a7a4"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-l4q5"
server_key: "d874dadb3e68126c734ad2d9bf563d15"
server_key_iv: "5d1392defb51832c604d232f57dc78d0"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.034992 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "83b6c6049e8542eca54337e30047a7a4"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-l4q5"
server_key: "d874dadb3e68126c734ad2d9bf563d15"
server_key_iv: "5d1392defb51832c604d232f57dc78d0"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.036229 19433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:19.037119 19447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:19.037289 19433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:19.037367 19433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "83b6c6049e8542eca54337e30047a7a4"
format_stamp: "Formatted at 2026-09-23 22:16:19 on dist-test-slave-l4q5"
server_key: "d874dadb3e68126c734ad2d9bf563d15"
server_key_iv: "5d1392defb51832c604d232f57dc78d0"
server_key_version: "encryptionkey@0"
I20260923 22:16:19.037436 19433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:19.047981 19433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:19.048362 19433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:19.048493 19433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:19.048710 19433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:19.049031 19433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:19.049065 19433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:19.049103 19433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:19.049125 19433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:19.056205 19433 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:33857
I20260923 22:16:19.056288 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:33857 every 8 connection(s)
I20260923 22:16:19.056643 19433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:19.063436 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.057643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:44113 (local address 127.1.132.190:34699)
0923 22:16:19.057707 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.057709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.057996 (+ 287us) server_negotiation.cc:407] Connection header received
0923 22:16:19.058880 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.058882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.058897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.058956 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:19.059827 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.060669 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.061748 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.062004 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.062080 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.063064 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:19.063065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:19.063066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:19.063071 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:19.063089 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:19.063093 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:19.063156 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.063331 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.063367 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:16:19.063673 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.057619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:34699 (local address 127.1.132.131:44113)
0923 22:16:19.057810 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.057847 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.058633 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.059080 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.059084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.059094 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:19.059706 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.059713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.060762 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.060763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.061643 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.061645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.062157 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.062158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.062294 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.062887 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:19.062888 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:19.062921 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:19.062953 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:19.063237 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:19.063243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:19.063247 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.063314 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.063444 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260923 22:16:19.064224 19561 heartbeater.cc:347] Connected to a master server at 127.1.132.190:34699
I20260923 22:16:19.064338 19561 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:19.064566 19561 heartbeater.cc:511] Master 127.1.132.190:34699 requested a full tablet report, sending...
I20260923 22:16:19.064571 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19433
I20260923 22:16:19.064642 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:19.064895 1554 external_mini_cluster.cc:1428] Setting key f25ef0f1144238465960f8f3957c173f
I20260923 22:16:19.065047 19111 ts_manager.cc:194] Registered new tserver with Master: 83b6c6049e8542eca54337e30047a7a4 (127.1.132.131:33857)
I20260923 22:16:19.065555 19111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.131:44113
I20260923 22:16:19.065800 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:19.069938 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.067088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.1.132.190:34699)
0923 22:16:19.067165 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.067167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.067232 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:16:19.067287 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.067287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.067297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.067346 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:19.067466 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.067938 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.068786 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.068989 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.069059 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.069696 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:19.069697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:19.069697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:19.069701 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:19.069715 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:19.069719 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:19.069761 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.069853 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.069878 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:16:19.071388 19110 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:19.072971 19110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:19.078985 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.075203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.1.132.131:33857)
0923 22:16:19.075450 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.075453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.075462 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:19.075501 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.075504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.075519 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.075600 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:19.075765 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.076949 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078070 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.078576 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078636 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.078721 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.078855 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.078898 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260923 22:16:19.078982 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.074998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:33857 (local address 127.0.0.1:59656)
0923 22:16:19.075154 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.075197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.075261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.075617 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.075620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.075622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:19.075692 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.075695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.077033 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.077034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.077978 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.077981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.078655 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.078739 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.078754 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.078833 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.078892 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260923 22:16:19.079782 19495 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 (DEFAULT_TABLE table=test-table [id=8b63f74f14a1418aac44966257b8a86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:19.080569 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.074998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:39409 (local address 127.0.0.1:40016)
0923 22:16:19.075380 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.075391 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.075445 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.075658 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.075658 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.075660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:19.075692 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.075693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.078068 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.078799 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.078801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.080317 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.080318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.080394 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.080398 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.080453 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.080494 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260923 22:16:19.080569 19495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec0a5f6daef4b4fb00d513964fafdb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:19.081007 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.075369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.1.132.129:39409)
0923 22:16:19.075529 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.075533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.075544 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:19.075579 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.075581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.075595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.075662 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:19.075744 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.076949 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078885 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.079922 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.080165 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.080287 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.080877 (+ 590us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.080925 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260923 22:16:19.081202 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.074999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:45595 (local address 127.0.0.1:38204)
0923 22:16:19.075533 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.075540 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.075588 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.076056 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.076057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.076059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:19.076098 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.076100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.078876 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.078877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.079930 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.079933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.080980 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.080982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.081042 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.081046 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.081093 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.081135 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":157,"thread_start_us":65,"threads_started":1}
I20260923 22:16:19.081429 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.075205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.1.132.130:45595)
0923 22:16:19.075376 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.075381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.075821 (+ 440us) server_negotiation.cc:407] Connection header received
0923 22:16:19.075867 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.075870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.075885 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.075954 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:19.076164 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.076956 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.080047 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.080781 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.080848 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.080931 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.081304 (+ 373us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.081350 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260923 22:16:19.083524 19575 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Bootstrap starting.
I20260923 22:16:19.084085 19363 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 (DEFAULT_TABLE table=test-table [id=8b63f74f14a1418aac44966257b8a86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:19.084282 19575 tablet_bootstrap.cc:654] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:19.084388 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 (DEFAULT_TABLE table=test-table [id=8b63f74f14a1418aac44966257b8a86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:19.084535 19363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec0a5f6daef4b4fb00d513964fafdb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:19.084573 19575 log.cc:824] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:19.084721 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec0a5f6daef4b4fb00d513964fafdb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:19.087831 19577 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Bootstrap starting.
I20260923 22:16:19.088332 19578 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Bootstrap starting.
I20260923 22:16:19.088652 19577 tablet_bootstrap.cc:654] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:19.088987 19577 log.cc:824] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:19.089058 19578 tablet_bootstrap.cc:654] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:19.089067 19575 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: No bootstrap required, opened a new log
I20260923 22:16:19.089133 19575 ts_tablet_manager.cc:1397] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260923 22:16:19.089331 19578 log.cc:824] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:19.090060 19578 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: No bootstrap required, opened a new log
I20260923 22:16:19.090135 19578 ts_tablet_manager.cc:1397] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:16:19.091403 19575 raft_consensus.cc:348] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.091557 19575 raft_consensus.cc:374] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:19.091578 19575 raft_consensus.cc:729] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b6c6049e8542eca54337e30047a7a4, State: Initialized, Role: FOLLOWER
I20260923 22:16:19.091682 19575 consensus_queue.cc:260] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.091951 19575 ts_tablet_manager.cc:1428] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:19.091967 19561 heartbeater.cc:503] Master 127.1.132.190:34699 was elected leader, sending a full tablet report...
I20260923 22:16:19.092145 19578 raft_consensus.cc:348] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.092320 19578 raft_consensus.cc:374] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:19.092376 19578 raft_consensus.cc:729] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbe7eebdd444853a22d494ff4a2438e, State: Initialized, Role: FOLLOWER
I20260923 22:16:19.092499 19578 consensus_queue.cc:260] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.092676 19577 tablet_bootstrap.cc:492] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: No bootstrap required, opened a new log
I20260923 22:16:19.092756 19578 ts_tablet_manager.cc:1428] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 22:16:19.092762 19577 ts_tablet_manager.cc:1397] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260923 22:16:19.092831 19299 heartbeater.cc:503] Master 127.1.132.190:34699 was elected leader, sending a full tablet report...
I20260923 22:16:19.094938 19577 raft_consensus.cc:348] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.095118 19577 raft_consensus.cc:374] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:19.095170 19577 raft_consensus.cc:729] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd59929d86c44259705223ef9bb7f4e, State: Initialized, Role: FOLLOWER
I20260923 22:16:19.095294 19577 consensus_queue.cc:260] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.095549 19577 ts_tablet_manager.cc:1428] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:19.095600 19430 heartbeater.cc:503] Master 127.1.132.190:34699 was elected leader, sending a full tablet report...
W20260923 22:16:19.181816 19431 tablet.cc:2369] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:19.301815 19300 tablet.cc:2369] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:19.307425 19562 tablet.cc:2369] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:19.374426 19582 raft_consensus.cc:482] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:19.374567 19582 raft_consensus.cc:504] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.374907 19582 leader_election.cc:288] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83b6c6049e8542eca54337e30047a7a4 (127.1.132.131:33857), bcd59929d86c44259705223ef9bb7f4e (127.1.132.130:45595)
I20260923 22:16:19.377894 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.375227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:53531 (local address 127.1.132.131:33857)
0923 22:16:19.375274 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.375275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.375394 (+ 119us) server_negotiation.cc:407] Connection header received
0923 22:16:19.375465 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.375466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.375475 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.375521 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:19.375675 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.376147 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377194 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.377565 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377630 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.377639 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.377777 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.377820 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:19.377939 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.375129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:33857 (local address 127.1.132.129:53531)
0923 22:16:19.375330 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.375338 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.375392 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.375548 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.375549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.375551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:19.375617 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.375619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.376270 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.376271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.377101 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.377104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.377655 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.377717 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.377721 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.377765 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.377812 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":62,"threads_started":1}
I20260923 22:16:19.377944 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.375199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:39937 (local address 127.1.132.130:45595)
0923 22:16:19.375251 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.375253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.375333 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:16:19.375389 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.375390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.375400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.375452 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:19.375598 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.376263 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377234 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.377612 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377674 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.377683 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.377832 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.377877 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:16:19.377964 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.375126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:45595 (local address 127.1.132.129:39937)
0923 22:16:19.375251 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.375261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:19.375330 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:19.375469 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:19.375469 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:19.375471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:19.375545 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.375547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.376376 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.376377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.377140 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:19.377143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.377708 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.377709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:19.377765 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.377769 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:19.377822 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:16:19.377863 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
I20260923 22:16:19.378357 19515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec0a5f6daef4b4fb00d513964fafdb4" candidate_uuid: "3dbe7eebdd444853a22d494ff4a2438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b6c6049e8542eca54337e30047a7a4" is_pre_election: true
I20260923 22:16:19.378360 19384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec0a5f6daef4b4fb00d513964fafdb4" candidate_uuid: "3dbe7eebdd444853a22d494ff4a2438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd59929d86c44259705223ef9bb7f4e" is_pre_election: true
I20260923 22:16:19.378525 19515 raft_consensus.cc:2454] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbe7eebdd444853a22d494ff4a2438e in term 0.
I20260923 22:16:19.378525 19384 raft_consensus.cc:2454] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbe7eebdd444853a22d494ff4a2438e in term 0.
I20260923 22:16:19.378773 19187 leader_election.cc:302] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3dbe7eebdd444853a22d494ff4a2438e, 83b6c6049e8542eca54337e30047a7a4, bcd59929d86c44259705223ef9bb7f4e; no voters:
I20260923 22:16:19.378963 19582 raft_consensus.cc:2781] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:19.379033 19582 raft_consensus.cc:482] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:19.379056 19582 raft_consensus.cc:3037] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:19.379778 19582 raft_consensus.cc:504] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.379966 19582 leader_election.cc:288] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [CANDIDATE]: Term 1 election: Requested vote from peers 83b6c6049e8542eca54337e30047a7a4 (127.1.132.131:33857), bcd59929d86c44259705223ef9bb7f4e (127.1.132.130:45595)
I20260923 22:16:19.380121 19384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec0a5f6daef4b4fb00d513964fafdb4" candidate_uuid: "3dbe7eebdd444853a22d494ff4a2438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd59929d86c44259705223ef9bb7f4e"
I20260923 22:16:19.380162 19515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec0a5f6daef4b4fb00d513964fafdb4" candidate_uuid: "3dbe7eebdd444853a22d494ff4a2438e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b6c6049e8542eca54337e30047a7a4"
I20260923 22:16:19.380252 19515 raft_consensus.cc:3037] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:19.380242 19384 raft_consensus.cc:3037] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:19.381137 19515 raft_consensus.cc:2454] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbe7eebdd444853a22d494ff4a2438e in term 1.
I20260923 22:16:19.381150 19384 raft_consensus.cc:2454] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbe7eebdd444853a22d494ff4a2438e in term 1.
I20260923 22:16:19.381382 19186 leader_election.cc:302] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbe7eebdd444853a22d494ff4a2438e, bcd59929d86c44259705223ef9bb7f4e; no voters:
I20260923 22:16:19.381469 19582 raft_consensus.cc:2781] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:19.381664 19582 raft_consensus.cc:686] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 1 LEADER]: Becoming Leader. State: Replica: 3dbe7eebdd444853a22d494ff4a2438e, State: Running, Role: LEADER
I20260923 22:16:19.381757 19582 consensus_queue.cc:237] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } }
I20260923 22:16:19.382718 19111 catalog_manager.cc:5495] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dbe7eebdd444853a22d494ff4a2438e (127.1.132.129). New cstate: current_term: 1 leader_uuid: "3dbe7eebdd444853a22d494ff4a2438e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dbe7eebdd444853a22d494ff4a2438e" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 39409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:19.437512 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.435012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40018 (local address 127.1.132.129:39409)
0923 22:16:19.435107 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:19.435109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:19.435118 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:19.435157 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:19.435158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:19.435177 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:19.435240 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:19.435398 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.435877 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.436634 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:19.436856 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:19.436924 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:19.437143 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:19.437255 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:19.437332 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:19.437428 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:16:19.437450 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 22:16:19.439303 19515 raft_consensus.cc:1261] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 1 FOLLOWER]: Refusing update from remote peer 3dbe7eebdd444853a22d494ff4a2438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:19.439359 19384 raft_consensus.cc:1261] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 1 FOLLOWER]: Refusing update from remote peer 3dbe7eebdd444853a22d494ff4a2438e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:19.439635 19586 consensus_queue.cc:1035] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6c6049e8542eca54337e30047a7a4" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:19.439749 19582 consensus_queue.cc:1035] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd59929d86c44259705223ef9bb7f4e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:19.442876 19591 mvcc.cc:205] Tried to move back new op lower bound from 7332666488580771840 to 7332666488348299264. Current Snapshot: MvccSnapshot[applied={T|T < 7332666488580771840}]
I20260923 22:16:19.443265 19590 mvcc.cc:205] Tried to move back new op lower bound from 7332666488580771840 to 7332666488348299264. Current Snapshot: MvccSnapshot[applied={T|T < 7332666488580771840}]
I20260923 22:16:19.445647 19111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:19.445840 19111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:19.447010 19111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 765a3522f8ad4d779a599eb6e4cd7183: Sending DeleteTablet for 3 replicas of tablet 0ec0a5f6daef4b4fb00d513964fafdb4
I20260923 22:16:19.447453 19233 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:19 UTC) from {username='slave'} at 127.0.0.1:40016
I20260923 22:16:19.447449 19495 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:19 UTC) from {username='slave'} at 127.0.0.1:59656
I20260923 22:16:19.447698 19363 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec0a5f6daef4b4fb00d513964fafdb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:19 UTC) from {username='slave'} at 127.0.0.1:38204
I20260923 22:16:19.447717 19600 tablet_replica.cc:330] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e: stopping tablet replica
I20260923 22:16:19.447824 19600 raft_consensus.cc:2229] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:19.447973 19600 raft_consensus.cc:2258] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 3dbe7eebdd444853a22d494ff4a2438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:19.447937 19601 tablet_replica.cc:330] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: stopping tablet replica
I20260923 22:16:19.448149 19599 tablet_replica.cc:330] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: stopping tablet replica
I20260923 22:16:19.448289 19599 raft_consensus.cc:2229] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:19.448356 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19171
I20260923 22:16:19.448415 19599 raft_consensus.cc:2258] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:19.449668 19601 raft_consensus.cc:2229] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:19.449819 19601 raft_consensus.cc:2258] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:19.449820 19599 ts_tablet_manager.cc:1905] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:19.450415 19601 ts_tablet_manager.cc:1905] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:19.452277 19601 ts_tablet_manager.cc:1918] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:19.452291 19599 ts_tablet_manager.cc:1918] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:19.452361 19601 log.cc:1196] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/0ec0a5f6daef4b4fb00d513964fafdb4
I20260923 22:16:19.452369 19599 log.cc:1196] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/0ec0a5f6daef4b4fb00d513964fafdb4
I20260923 22:16:19.452631 19601 ts_tablet_manager.cc:1939] T 0ec0a5f6daef4b4fb00d513964fafdb4 P bcd59929d86c44259705223ef9bb7f4e: Deleting consensus metadata
I20260923 22:16:19.452643 19599 ts_tablet_manager.cc:1939] T 0ec0a5f6daef4b4fb00d513964fafdb4 P 83b6c6049e8542eca54337e30047a7a4: Deleting consensus metadata
I20260923 22:16:19.453212 19096 catalog_manager.cc:4841] TS 83b6c6049e8542eca54337e30047a7a4 (127.1.132.131:33857): tablet 0ec0a5f6daef4b4fb00d513964fafdb4 (table test-table [id=8b63f74f14a1418aac44966257b8a86d]) successfully deleted
I20260923 22:16:19.453336 19095 catalog_manager.cc:4841] TS bcd59929d86c44259705223ef9bb7f4e (127.1.132.130:45595): tablet 0ec0a5f6daef4b4fb00d513964fafdb4 (table test-table [id=8b63f74f14a1418aac44966257b8a86d]) successfully deleted
W20260923 22:16:19.454107 19098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:39409 (error 108)
I20260923 22:16:19.454569 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19302
I20260923 22:16:19.454595 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:19.454454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:39409 (local address 127.0.0.1:40032)
0923 22:16:19.454515 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:19.454561 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:39409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:16:19.456502 19098 catalog_manager.cc:4584] TS 3dbe7eebdd444853a22d494ff4a2438e (127.1.132.129:39409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ec0a5f6daef4b4fb00d513964fafdb4: Network error: Client connection negotiation failed: client connection to 127.1.132.129:39409: connect: Connection refused (error 111)
I20260923 22:16:19.460062 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19433
I20260923 22:16:19.465847 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19080
2026-09-23T22:16:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1013 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:19 dist-test-slave-l4q5 krb5kdc[19605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:19 dist-test-slave-l4q5 krb5kdc[19605](info): set up 2 sockets
Sep 23 22:16:19 dist-test-slave-l4q5 krb5kdc[19605](info): commencing operation
krb5kdc: starting...
W20260923 22:16:20.451743 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:20 dist-test-slave-l4q5 krb5kdc[19605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:20.571794 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:41505
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:41505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:20.652613 19618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:20.652848 19618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:20.652882 19618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:20.654167 19618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:20.654230 19618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:20.654249 19618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:20.654266 19618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:20.654281 19618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:20.655961 19618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:41505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:41505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:20.656286 19618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:20.656512 19618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:20.662082 19618 server_base.cc:1034] running on GCE node
W20260923 22:16:20.661996 19626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:20.662045 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:20.661993 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:20.662554 19618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:20.662875 19618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:20.664110 19618 hybrid_clock.cc:648] HybridClock initialized: now 1790201780664092 us; error 29 us; skew 500 ppm
Sep 23 22:16:20 dist-test-slave-l4q5 krb5kdc[19605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:20.672276 19618 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:20.672744 19618 webserver.cc:460] Webserver started at http://127.1.132.190:34139/ using document root <none> and password file <none>
I20260923 22:16:20.672943 19618 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:20.672986 19618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:20.673089 19618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:20.674192 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "e524a436b09442c1bf47e30099a14916"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "5175214825c04a29e761efa3467e1218"
server_key_iv: "f4f87a7a8983b826c100d9b1a95fb640"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.674510 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "e524a436b09442c1bf47e30099a14916"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "5175214825c04a29e761efa3467e1218"
server_key_iv: "f4f87a7a8983b826c100d9b1a95fb640"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.675868 19618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:20.676680 19633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.676910 19618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:16:20.677011 19618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "e524a436b09442c1bf47e30099a14916"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "5175214825c04a29e761efa3467e1218"
server_key_iv: "f4f87a7a8983b826c100d9b1a95fb640"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.677075 19618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:20.693074 19618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:20.695487 19618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:20.695618 19618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:20.699496 19618 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:41505
I20260923 22:16:20.699544 19685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:41505 every 8 connection(s)
I20260923 22:16:20.699870 19618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:20.700780 19686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:20.703032 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: Bootstrap starting.
I20260923 22:16:20.703589 19686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:20.703841 19686 log.cc:824] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:20.704609 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: No bootstrap required, opened a new log
I20260923 22:16:20.706233 19686 raft_consensus.cc:348] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } }
I20260923 22:16:20.706385 19686 raft_consensus.cc:374] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:20.706418 19686 raft_consensus.cc:729] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e524a436b09442c1bf47e30099a14916, State: Initialized, Role: FOLLOWER
I20260923 22:16:20.706557 19686 consensus_queue.cc:260] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } }
I20260923 22:16:20.706622 19686 raft_consensus.cc:388] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:20.706652 19686 raft_consensus.cc:482] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:20.706699 19686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:20.706876 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19618
I20260923 22:16:20.706969 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:20.707129 1554 external_mini_cluster.cc:1428] Setting key 7b5f0b620fea6003cd4bc5896c543832
I20260923 22:16:20.707325 19686 raft_consensus.cc:504] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } }
I20260923 22:16:20.707422 19686 leader_election.cc:302] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e524a436b09442c1bf47e30099a14916; no voters:
I20260923 22:16:20.707597 19686 leader_election.cc:288] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:20.707775 19691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:20.707894 19686 sys_catalog.cc:564] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:20.707985 19691 raft_consensus.cc:686] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [term 1 LEADER]: Becoming Leader. State: Replica: e524a436b09442c1bf47e30099a14916, State: Running, Role: LEADER
I20260923 22:16:20.708128 19691 consensus_queue.cc:237] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } }
I20260923 22:16:20.708602 19693 sys_catalog.cc:455] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e524a436b09442c1bf47e30099a14916. Latest consensus state: current_term: 1 leader_uuid: "e524a436b09442c1bf47e30099a14916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } } }
I20260923 22:16:20.708680 19693 sys_catalog.cc:458] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:20.708884 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e524a436b09442c1bf47e30099a14916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e524a436b09442c1bf47e30099a14916" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 41505 } } }
I20260923 22:16:20.709208 19700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:20.709208 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:20.709892 19700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:20.711447 19700 catalog_manager.cc:1347] Generated new cluster ID: 0c8f685a718e48dab020ef3db5d1407c
I20260923 22:16:20.711499 19700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:20 dist-test-slave-l4q5 krb5kdc[19605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:20.718504 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:20.707637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.1.132.190:41505)
0923 22:16:20.707847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:20.707858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:20.707894 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:16:20.708021 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:20.708029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:20.708090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:20.708259 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:20.709074 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.710439 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.711335 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.711935 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.712044 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:20.714836 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:20.714841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:20.714841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:20.714854 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:20.716815 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.717254 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.717256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.717257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.717374 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.717567 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.717567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.717568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.717868 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:20.718025 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:20.718137 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:16:20.718272 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260923 22:16:20.750681 19700 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:20.751291 19700 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:20.762249 19700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: Generated new TSK 0
I20260923 22:16:20.762454 19700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:16:20.800233 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:20.880987 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:20.881192 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:20.881217 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:20.882373 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:20.882412 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:20.882437 19712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:20.883843 19712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:20.884032 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:20.884245 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:20.889061 19720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:20.889103 19712 server_base.cc:1034] running on GCE node
W20260923 22:16:20.889055 19717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:20.889055 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:20.889487 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:20.889719 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:20.890875 19712 hybrid_clock.cc:648] HybridClock initialized: now 1790201780890860 us; error 31 us; skew 500 ppm
Sep 23 22:16:20 dist-test-slave-l4q5 krb5kdc[19605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:20.893532 19712 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:16:20.893967 19712 webserver.cc:460] Webserver started at http://127.1.132.129:37097/ using document root <none> and password file <none>
I20260923 22:16:20.894199 19712 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:20.894244 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:20.894374 19712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:20.895447 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "e0f75196c74d067f08bdd5a100d73807"
server_key_iv: "353db60551e67b7791f392642c05a82d"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.895781 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "e0f75196c74d067f08bdd5a100d73807"
server_key_iv: "353db60551e67b7791f392642c05a82d"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.897164 19712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:20.897850 19727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.897987 19712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.898072 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4"
format_stamp: "Formatted at 2026-09-23 22:16:20 on dist-test-slave-l4q5"
server_key: "e0f75196c74d067f08bdd5a100d73807"
server_key_iv: "353db60551e67b7791f392642c05a82d"
server_key_version: "encryptionkey@0"
I20260923 22:16:20.898137 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:20.922309 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:20.924100 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:20.924263 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:20.924495 19712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:20.924784 19712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:20.924818 19712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.924847 19712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:20.924870 19712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:20.930262 19712 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:45187
I20260923 22:16:20.930328 19840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:45187 every 8 connection(s)
I20260923 22:16:20.930629 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:20.935285 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19712
I20260923 22:16:20.935388 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:20.935588 1554 external_mini_cluster.cc:1428] Setting key cadd7bbced672c552297ff8b2afd122d
Sep 23 22:16:20 dist-test-slave-l4q5 krb5kdc[19605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201780, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:20.939987 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:20.931353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:55587 (local address 127.1.132.190:41505)
0923 22:16:20.931428 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:20.931430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:20.931538 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:16:20.932083 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:20.932084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:20.932099 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:20.932144 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:20.933086 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.933593 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.934580 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.934783 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.934856 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:20.937220 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:20.937221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:20.937221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:20.937229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:20.938631 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.939061 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.939062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.939062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.939119 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:20.939339 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:20.939341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:20.939341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:20.939461 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:20.939520 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:20.939797 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 22:16:20.939895 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:16:20.940158 19843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:20.931279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:41505 (local address 127.1.132.129:55587)
0923 22:16:20.931424 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:20.931462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:20.931998 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:20.932235 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:20.932239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:20.932793 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:20.932995 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:20.933000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.933663 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.933663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:20.934500 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:20.934502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:20.934883 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:20.934883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:20.934988 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:20.935507 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:20.935517 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:20.937122 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:20.938723 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:20.938730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:20.938732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:20.938978 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:20.939174 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:20.939175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:20.939175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:20.939281 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:20.939517 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:20.939525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:20.939669 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 22:16:20.939760 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:16:20.939938 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260923 22:16:20.940727 19841 heartbeater.cc:347] Connected to a master server at 127.1.132.190:41505
I20260923 22:16:20.940840 19841 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:20.941041 19841 heartbeater.cc:511] Master 127.1.132.190:41505 requested a full tablet report, sending...
I20260923 22:16:20.941619 19650 ts_manager.cc:194] Registered new tserver with Master: 650b233aa0544c5ab04b6bb8c2d8e4b4 (127.1.132.129:45187)
I20260923 22:16:20.942531 19650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:55587
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:16:20.963878 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:21.044138 19846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.044366 19846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.044389 19846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.045535 19846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.045572 19846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.045598 19846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:21.047021 19846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.047240 19846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.047441 19846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:21.052785 19852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.052873 19854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.052784 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.052815 19846 server_base.cc:1034] running on GCE node
I20260923 22:16:21.053231 19846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.053457 19846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.054626 19846 hybrid_clock.cc:648] HybridClock initialized: now 1790201781054607 us; error 30 us; skew 500 ppm
Sep 23 22:16:21 dist-test-slave-l4q5 krb5kdc[19605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:21.057304 19846 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:16:21.057705 19846 webserver.cc:460] Webserver started at http://127.1.132.130:40805/ using document root <none> and password file <none>
I20260923 22:16:21.057910 19846 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.057950 19846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.058043 19846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.059072 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "3c7191e66aac457c9d370585a0330d43"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a2f3b801b3f3b723dbd70932b5471c55"
server_key_iv: "609d7dfb07623f94aaf0caa9d91e67e0"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.059357 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "3c7191e66aac457c9d370585a0330d43"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a2f3b801b3f3b723dbd70932b5471c55"
server_key_iv: "609d7dfb07623f94aaf0caa9d91e67e0"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.060554 19846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:21.061273 19861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.061401 19846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:21.061480 19846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "3c7191e66aac457c9d370585a0330d43"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a2f3b801b3f3b723dbd70932b5471c55"
server_key_iv: "609d7dfb07623f94aaf0caa9d91e67e0"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.061550 19846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.079983 19846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.082000 19846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.082125 19846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.082361 19846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:21.082653 19846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:21.082686 19846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.082707 19846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:21.082718 19846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.088836 19846 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:45939
I20260923 22:16:21.088871 19974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:45939 every 8 connection(s)
I20260923 22:16:21.089260 19846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:16:21 dist-test-slave-l4q5 krb5kdc[19605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:21.098450 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.089995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:47653 (local address 127.1.132.190:41505)
0923 22:16:21.090058 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.090060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.090242 (+ 182us) server_negotiation.cc:407] Connection header received
0923 22:16:21.090710 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.090711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.090720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.090826 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.091755 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.092353 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.093369 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.093569 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.093635 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.095886 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.095888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.095889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:21.095897 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.097198 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.097609 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.097610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.097610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.097657 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.097852 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.097852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.097852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.097956 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.098000 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.098288 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.098381 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:16:21.098567 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.089971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:41505 (local address 127.1.132.130:47653)
0923 22:16:21.090124 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.090165 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.090636 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.090883 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.090886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.091436 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:21.091630 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.091635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.092436 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.092438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.093268 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.093272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.093668 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.093669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.093789 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.094343 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:21.094352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:21.095770 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:21.097283 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:21.097290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:21.097292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:21.097545 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:21.097703 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:21.097703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:21.097704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:21.097803 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:21.097997 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:21.098000 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:21.098133 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.098225 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.098374 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260923 22:16:21.099023 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19846
I20260923 22:16:21.099086 19975 heartbeater.cc:347] Connected to a master server at 127.1.132.190:41505
I20260923 22:16:21.099138 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:21.099196 19975 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:21.099328 1554 external_mini_cluster.cc:1428] Setting key 88d9922b99d99d09f1fd23189f6d367f
I20260923 22:16:21.099413 19975 heartbeater.cc:511] Master 127.1.132.190:41505 requested a full tablet report, sending...
I20260923 22:16:21.099817 19649 ts_manager.cc:194] Registered new tserver with Master: 3c7191e66aac457c9d370585a0330d43 (127.1.132.130:45939)
I20260923 22:16:21.100353 19649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:47653
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:16:21.124276 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:21.204706 19980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.205008 19980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.205045 19980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.206851 19980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.206907 19980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.206944 19980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:21.209662 19980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:41505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.210836 19980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.211153 19980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:21.218086 19988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.218093 19985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.218093 19986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.218443 19980 server_base.cc:1034] running on GCE node
I20260923 22:16:21.218674 19980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.218918 19980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.220084 19980 hybrid_clock.cc:648] HybridClock initialized: now 1790201781220074 us; error 33 us; skew 500 ppm
Sep 23 22:16:21 dist-test-slave-l4q5 krb5kdc[19605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:21.222644 19980 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:16:21.223124 19980 webserver.cc:460] Webserver started at http://127.1.132.131:32769/ using document root <none> and password file <none>
I20260923 22:16:21.223336 19980 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.223385 19980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.223507 19980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.224607 19980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "26b438fab49a4d658c259dfe595f788d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "026c42912b8461ae3a911162d0f6073e"
server_key_iv: "c92b5f8bb901354308953a60888e0c08"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.224953 19980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "26b438fab49a4d658c259dfe595f788d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "026c42912b8461ae3a911162d0f6073e"
server_key_iv: "c92b5f8bb901354308953a60888e0c08"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.226181 19980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:21.227058 19995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.227252 19980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:21.227342 19980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "26b438fab49a4d658c259dfe595f788d"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "026c42912b8461ae3a911162d0f6073e"
server_key_iv: "c92b5f8bb901354308953a60888e0c08"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.227411 19980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.241369 19980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.243495 19980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.243631 19980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.243852 19980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:21.244145 19980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:21.244247 19980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.244287 19980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:21.244304 19980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.249764 19980 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:42461
I20260923 22:16:21.249814 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:42461 every 8 connection(s)
I20260923 22:16:21.250118 19980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:21.254347 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 19980
I20260923 22:16:21.254434 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:21.254583 1554 external_mini_cluster.cc:1428] Setting key 284668bb01ae4b8410bb3b48fadc2d14
Sep 23 22:16:21 dist-test-slave-l4q5 krb5kdc[19605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201781, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:21.258929 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.250794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:37195 (local address 127.1.132.190:41505)
0923 22:16:21.250863 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.250865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.251035 (+ 170us) server_negotiation.cc:407] Connection header received
0923 22:16:21.251494 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.251495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.251505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.251560 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.252485 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.252975 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.253909 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.254085 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.254128 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.256463 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.256465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.256465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:21.256472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.257681 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.258071 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.258072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.258072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.258127 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.258375 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.258376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.258377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.258504 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.258564 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.258786 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.258853 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:16:21.259111 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.250750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:41505 (local address 127.1.132.131:37195)
0923 22:16:21.250914 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.250956 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.251423 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.251618 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.251621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.252160 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:21.252406 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.252411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.253038 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.253039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.253849 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.253851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.254137 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.254137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.254233 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.254761 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:21.254770 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:21.256382 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:21.257749 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:21.257754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:21.257756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:21.258001 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:21.258179 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:21.258179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:21.258180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:21.258278 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:21.258560 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:21.258565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:21.258683 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.258764 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.258896 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260923 22:16:21.259518 20109 heartbeater.cc:347] Connected to a master server at 127.1.132.190:41505
I20260923 22:16:21.259596 20109 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:21.259752 20109 heartbeater.cc:511] Master 127.1.132.190:41505 requested a full tablet report, sending...
I20260923 22:16:21.260097 19649 ts_manager.cc:194] Registered new tserver with Master: 26b438fab49a4d658c259dfe595f788d (127.1.132.131:42461)
I20260923 22:16:21.260491 19649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:37195
I20260923 22:16:21.266359 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:21.273523 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.267611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.1.132.190:41505)
0923 22:16:21.267660 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.267663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.267766 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:16:21.267840 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.267841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.267851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.267897 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.268462 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.268939 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.269621 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.269808 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.269860 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.271121 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.271122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.271123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:21.271130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.272406 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.272782 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.272784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.272784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.272836 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:21.273024 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:21.273024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:21.273025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:21.273162 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.273256 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.273376 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.273434 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:16:21.275456 19649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:21.277101 19649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:21.283553 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:45939 (local address 127.0.0.1:60356)
0923 22:16:21.279792 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.279801 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.279856 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.280021 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.280024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.280026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.280127 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.280129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281240 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.281241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.282448 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.282451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.283222 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.283223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.283329 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.283345 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.283407 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.283467 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":80,"threads_started":1}
I20260923 22:16:21.283564 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.1.132.130:45939)
0923 22:16:21.279806 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.279810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.279846 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:16:21.279915 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.279916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.279928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.279987 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.280227 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281143 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.282533 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.283148 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.283210 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.283273 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.283432 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.283477 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260923 22:16:21.283849 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:42461 (local address 127.0.0.1:60342)
0923 22:16:21.279633 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.279674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.279738 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.280130 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.280130 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.280132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.280208 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.280211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281562 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.281563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.282624 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.282626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.283616 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.283617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.283679 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.283684 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.283734 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.283777 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":38,"thread_start_us":93,"threads_started":1}
I20260923 22:16:21.284082 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.1.132.131:42461)
0923 22:16:21.279978 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.279983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.279992 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:21.280026 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.280028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.280042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.280123 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.280309 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281457 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.282711 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.283458 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.283527 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.283580 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.283955 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.284007 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":87,"threads_started":1}
I20260923 22:16:21.284379 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45187 (local address 127.0.0.1:39582)
0923 22:16:21.279635 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.279673 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.279737 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.280533 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.280534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.280537 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.280574 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.280575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281751 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.281752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.282702 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.282706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.284055 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.284056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.284116 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.284221 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.284275 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.284317 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260923 22:16:21.284343 19909 tablet_service.cc:1467] Processing CreateTablet for tablet f6787908fd854c64b197b136c0456280 (DEFAULT_TABLE table=test-table [id=2430b82226864f2e834879bc5a528cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.284735 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.279421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.1.132.129:45187)
0923 22:16:21.280297 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.280301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.280310 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:21.280345 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.280347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.280360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.280451 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.280652 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.281668 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.282778 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.283458 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.283527 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.283580 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.284611 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.284659 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":74,"threads_started":1}
I20260923 22:16:21.284750 19909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6787908fd854c64b197b136c0456280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.284791 20043 tablet_service.cc:1467] Processing CreateTablet for tablet f6787908fd854c64b197b136c0456280 (DEFAULT_TABLE table=test-table [id=2430b82226864f2e834879bc5a528cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.285113 20043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6787908fd854c64b197b136c0456280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.285665 19775 tablet_service.cc:1467] Processing CreateTablet for tablet f6787908fd854c64b197b136c0456280 (DEFAULT_TABLE table=test-table [id=2430b82226864f2e834879bc5a528cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.285974 19775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6787908fd854c64b197b136c0456280. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.287771 20126 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Bootstrap starting.
I20260923 22:16:21.288347 20126 tablet_bootstrap.cc:654] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.288306 20127 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Bootstrap starting.
I20260923 22:16:21.288909 20126 log.cc:824] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.289009 20127 tablet_bootstrap.cc:654] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.289008 20128 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: Bootstrap starting.
I20260923 22:16:21.289484 20127 log.cc:824] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.289604 20128 tablet_bootstrap.cc:654] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.289690 20126 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: No bootstrap required, opened a new log
I20260923 22:16:21.289774 20126 ts_tablet_manager.cc:1397] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:16:21.290117 20127 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: No bootstrap required, opened a new log
I20260923 22:16:21.290117 20128 log.cc:824] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.290176 20127 ts_tablet_manager.cc:1397] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:16:21.290809 20128 tablet_bootstrap.cc:492] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: No bootstrap required, opened a new log
I20260923 22:16:21.290871 20128 ts_tablet_manager.cc:1397] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:21.291692 20126 raft_consensus.cc:348] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.291781 20127 raft_consensus.cc:348] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.291894 20126 raft_consensus.cc:374] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.291923 20127 raft_consensus.cc:374] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.291932 20126 raft_consensus.cc:729] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7191e66aac457c9d370585a0330d43, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.291960 20127 raft_consensus.cc:729] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b438fab49a4d658c259dfe595f788d, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.292045 20126 consensus_queue.cc:260] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.292088 20127 consensus_queue.cc:260] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.292351 20126 ts_tablet_manager.cc:1428] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:21.292369 20127 ts_tablet_manager.cc:1428] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:16:21.292480 20109 heartbeater.cc:503] Master 127.1.132.190:41505 was elected leader, sending a full tablet report...
I20260923 22:16:21.292529 19975 heartbeater.cc:503] Master 127.1.132.190:41505 was elected leader, sending a full tablet report...
I20260923 22:16:21.292799 20128 raft_consensus.cc:348] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.292969 20128 raft_consensus.cc:374] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.293004 20128 raft_consensus.cc:729] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650b233aa0544c5ab04b6bb8c2d8e4b4, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.293092 20128 consensus_queue.cc:260] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.293367 20128 ts_tablet_manager.cc:1428] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:16:21.293395 19841 heartbeater.cc:503] Master 127.1.132.190:41505 was elected leader, sending a full tablet report...
I20260923 22:16:21.327862 20132 raft_consensus.cc:482] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:21.328006 20132 raft_consensus.cc:504] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.328450 20132 leader_election.cc:288] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 650b233aa0544c5ab04b6bb8c2d8e4b4 (127.1.132.129:45187), 26b438fab49a4d658c259dfe595f788d (127.1.132.131:42461)
I20260923 22:16:21.331660 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.328776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:39359 (local address 127.1.132.131:42461)
0923 22:16:21.328824 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.328826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.328836 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:21.328866 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.328867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.328878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.328947 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.329100 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.329634 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.330718 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.331294 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.331370 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.331380 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.331543 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.331582 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:21.331660 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.328645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:42461 (local address 127.1.132.130:39359)
0923 22:16:21.328707 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.328720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.328776 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.328965 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.328966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.328968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.329042 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.329044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.329767 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.329769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.330627 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.330631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.331383 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.331384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.331475 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.331478 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.331525 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.331574 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:16:21.331939 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.328726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:49933 (local address 127.1.132.129:45187)
0923 22:16:21.328773 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.328776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.329008 (+ 232us) server_negotiation.cc:407] Connection header received
0923 22:16:21.329059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.329060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.329069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.329115 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.329294 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.329787 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.330871 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.331530 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.331617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.331627 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.331811 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.331870 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:21.332064 20063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6787908fd854c64b197b136c0456280" candidate_uuid: "3c7191e66aac457c9d370585a0330d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b438fab49a4d658c259dfe595f788d" is_pre_election: true
I20260923 22:16:21.332151 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.328646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45187 (local address 127.1.132.130:49933)
0923 22:16:21.328955 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.328963 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.329011 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.329138 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.329139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.329141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.329225 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.329227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.329863 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.329864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.330785 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.330788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.331638 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.331639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.331710 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.331714 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.332025 (+ 311us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.332076 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":76,"thread_start_us":185,"threads_started":1}
I20260923 22:16:21.332265 20063 raft_consensus.cc:2454] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7191e66aac457c9d370585a0330d43 in term 0.
I20260923 22:16:21.332533 19862 leader_election.cc:302] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26b438fab49a4d658c259dfe595f788d, 3c7191e66aac457c9d370585a0330d43; no voters:
I20260923 22:16:21.332535 19795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6787908fd854c64b197b136c0456280" candidate_uuid: "3c7191e66aac457c9d370585a0330d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" is_pre_election: true
I20260923 22:16:21.332662 19795 raft_consensus.cc:2454] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7191e66aac457c9d370585a0330d43 in term 0.
I20260923 22:16:21.332703 20132 raft_consensus.cc:2781] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:21.332792 20132 raft_consensus.cc:482] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:21.332821 20132 raft_consensus.cc:3037] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:21.333573 20132 raft_consensus.cc:504] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.333709 20132 leader_election.cc:288] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [CANDIDATE]: Term 1 election: Requested vote from peers 650b233aa0544c5ab04b6bb8c2d8e4b4 (127.1.132.129:45187), 26b438fab49a4d658c259dfe595f788d (127.1.132.131:42461)
I20260923 22:16:21.333850 20063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6787908fd854c64b197b136c0456280" candidate_uuid: "3c7191e66aac457c9d370585a0330d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26b438fab49a4d658c259dfe595f788d"
I20260923 22:16:21.333849 19795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6787908fd854c64b197b136c0456280" candidate_uuid: "3c7191e66aac457c9d370585a0330d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4"
I20260923 22:16:21.333921 19795 raft_consensus.cc:3037] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:21.333923 20063 raft_consensus.cc:3037] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:21.334652 20063 raft_consensus.cc:2454] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7191e66aac457c9d370585a0330d43 in term 1.
I20260923 22:16:21.334653 19795 raft_consensus.cc:2454] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7191e66aac457c9d370585a0330d43 in term 1.
I20260923 22:16:21.334859 19863 leader_election.cc:302] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7191e66aac457c9d370585a0330d43, 650b233aa0544c5ab04b6bb8c2d8e4b4; no voters:
I20260923 22:16:21.334951 20132 raft_consensus.cc:2781] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:21.335114 20132 raft_consensus.cc:686] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 1 LEADER]: Becoming Leader. State: Replica: 3c7191e66aac457c9d370585a0330d43, State: Running, Role: LEADER
I20260923 22:16:21.335193 20132 consensus_queue.cc:237] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } }
I20260923 22:16:21.336135 19650 catalog_manager.cc:5495] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c7191e66aac457c9d370585a0330d43 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "3c7191e66aac457c9d370585a0330d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c7191e66aac457c9d370585a0330d43" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 45939 } health_report { overall_health: HEALTHY } } }
W20260923 22:16:21.339880 19976 tablet.cc:2369] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:21.341658 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.339453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.1.132.130:45939)
0923 22:16:21.339495 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.339497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.339509 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:16:21.339583 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.339584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.339599 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.339647 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:21.339734 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.340260 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.340983 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.341181 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.341236 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.341375 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:21.341457 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:21.341523 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.341557 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.341565 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:16:21.343384 20063 raft_consensus.cc:1261] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 1 FOLLOWER]: Refusing update from remote peer 3c7191e66aac457c9d370585a0330d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:21.343385 19795 raft_consensus.cc:1261] T f6787908fd854c64b197b136c0456280 P 650b233aa0544c5ab04b6bb8c2d8e4b4 [term 1 FOLLOWER]: Refusing update from remote peer 3c7191e66aac457c9d370585a0330d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:21.343725 20136 consensus_queue.cc:1035] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26b438fab49a4d658c259dfe595f788d" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 42461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:21.343847 20132 consensus_queue.cc:1035] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "650b233aa0544c5ab04b6bb8c2d8e4b4" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 45187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:21.346938 20141 mvcc.cc:205] Tried to move back new op lower bound from 7332666496379944960 to 7332666496349683712. Current Snapshot: MvccSnapshot[applied={T|T < 7332666496379944960}]
I20260923 22:16:21.348871 19650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:21.348999 19650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:21.349759 19650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e524a436b09442c1bf47e30099a14916: Sending DeleteTablet for 3 replicas of tablet f6787908fd854c64b197b136c0456280
I20260923 22:16:21.350116 20043 tablet_service.cc:1514] Processing DeleteTablet for tablet f6787908fd854c64b197b136c0456280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:21 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:60342
I20260923 22:16:21.350250 19775 tablet_service.cc:1514] Processing DeleteTablet for tablet f6787908fd854c64b197b136c0456280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:21 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:39582
I20260923 22:16:21.350351 20148 tablet_replica.cc:330] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: stopping tablet replica
I20260923 22:16:21.350468 20148 raft_consensus.cc:2229] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:21.350540 19909 tablet_service.cc:1514] Processing DeleteTablet for tablet f6787908fd854c64b197b136c0456280 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:21 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:60356
I20260923 22:16:21.350579 20148 raft_consensus.cc:2258] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:21.350603 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19712
I20260923 22:16:21.352284 20148 ts_tablet_manager.cc:1905] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:21.353264 20150 tablet_replica.cc:330] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: stopping tablet replica
I20260923 22:16:21.353384 20150 raft_consensus.cc:2229] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:21.353519 20150 raft_consensus.cc:2258] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:21.353922 20150 ts_tablet_manager.cc:1905] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:21.354099 20148 ts_tablet_manager.cc:1918] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:21.354180 20148 log.cc:1196] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/f6787908fd854c64b197b136c0456280
I20260923 22:16:21.354456 20148 ts_tablet_manager.cc:1939] T f6787908fd854c64b197b136c0456280 P 26b438fab49a4d658c259dfe595f788d: Deleting consensus metadata
I20260923 22:16:21.355000 19634 catalog_manager.cc:4841] TS 26b438fab49a4d658c259dfe595f788d (127.1.132.131:42461): tablet f6787908fd854c64b197b136c0456280 (table test-table [id=2430b82226864f2e834879bc5a528cc8]) successfully deleted
I20260923 22:16:21.355782 20150 ts_tablet_manager.cc:1918] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:21.355922 20150 log.cc:1196] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/f6787908fd854c64b197b136c0456280
I20260923 22:16:21.356245 20150 ts_tablet_manager.cc:1939] T f6787908fd854c64b197b136c0456280 P 3c7191e66aac457c9d370585a0330d43: Deleting consensus metadata
I20260923 22:16:21.356679 19635 catalog_manager.cc:4841] TS 3c7191e66aac457c9d370585a0330d43 (127.1.132.130:45939): tablet f6787908fd854c64b197b136c0456280 (table test-table [id=2430b82226864f2e834879bc5a528cc8]) successfully deleted
W20260923 22:16:21.357239 19635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:45187 (error 108)
I20260923 22:16:21.357761 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.357655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:45187 (local address 127.0.0.1:39592)
0923 22:16:21.357694 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.357729 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:45187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 22:16:21.357759 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19846
W20260923 22:16:21.357867 19635 catalog_manager.cc:4584] TS 650b233aa0544c5ab04b6bb8c2d8e4b4 (127.1.132.129:45187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f6787908fd854c64b197b136c0456280: Network error: Client connection negotiation failed: client connection to 127.1.132.129:45187: connect: Connection refused (error 111)
I20260923 22:16:21.362149 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19980
I20260923 22:16:21.366904 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 19618
2026-09-23T22:16:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1901 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T22:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:21Z Disabled control of system clock
I20260923 22:16:21.393028 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33919
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:33919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:21.478411 20155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.478672 20155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.478694 20155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.480223 20155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:21.480283 20155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.480299 20155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.480315 20155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:21.480331 20155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:21.481966 20155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:33919
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.482268 20155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.482573 20155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:21.487879 20161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.487927 20155 server_base.cc:1034] running on GCE node
W20260923 22:16:21.488056 20163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.487931 20160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.488405 20155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.488703 20155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.489872 20155 hybrid_clock.cc:648] HybridClock initialized: now 1790201781489834 us; error 52 us; skew 500 ppm
I20260923 22:16:21.491204 20155 webserver.cc:460] Webserver started at http://127.1.132.190:43389/ using document root <none> and password file <none>
I20260923 22:16:21.491448 20155 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.491495 20155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.491623 20155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.492751 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "e9e363300e1f40bca9dcf0a5098ab3ff"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "072886472dc8f0a015a4c062792dcc44"
server_key_iv: "ab9f6f8c3756d9fee56cfd372bbade63"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.493085 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "e9e363300e1f40bca9dcf0a5098ab3ff"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "072886472dc8f0a015a4c062792dcc44"
server_key_iv: "ab9f6f8c3756d9fee56cfd372bbade63"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.494438 20155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:21.495262 20169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.495560 20155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:21.495647 20155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "e9e363300e1f40bca9dcf0a5098ab3ff"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "072886472dc8f0a015a4c062792dcc44"
server_key_iv: "ab9f6f8c3756d9fee56cfd372bbade63"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.495728 20155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.519273 20155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.519619 20155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.519737 20155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.523355 20155 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:33919
I20260923 22:16:21.523401 20221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:33919 every 8 connection(s)
I20260923 22:16:21.523722 20155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:21.524713 20222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.527065 20222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: Bootstrap starting.
I20260923 22:16:21.527623 20222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.527925 20222 log.cc:824] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.528041 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20155
I20260923 22:16:21.528147 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:21.528340 1554 external_mini_cluster.cc:1428] Setting key 2d02ac6d07e2da8a3f8eea485307e66e
I20260923 22:16:21.528694 20222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: No bootstrap required, opened a new log
I20260923 22:16:21.530761 20222 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } }
I20260923 22:16:21.530910 20222 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.530946 20222 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e363300e1f40bca9dcf0a5098ab3ff, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.531070 20222 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } }
I20260923 22:16:21.531141 20222 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:21.531180 20222 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:21.531226 20222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:21.531919 20222 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } }
I20260923 22:16:21.532028 20222 leader_election.cc:302] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9e363300e1f40bca9dcf0a5098ab3ff; no voters:
I20260923 22:16:21.532222 20222 leader_election.cc:288] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:21.532259 20227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:21.532431 20227 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [term 1 LEADER]: Becoming Leader. State: Replica: e9e363300e1f40bca9dcf0a5098ab3ff, State: Running, Role: LEADER
I20260923 22:16:21.532493 20222 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:21.532523 20227 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } }
I20260923 22:16:21.532805 20227 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9e363300e1f40bca9dcf0a5098ab3ff. Latest consensus state: current_term: 1 leader_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } } }
I20260923 22:16:21.532931 20227 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:21.532819 20229 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e363300e1f40bca9dcf0a5098ab3ff" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33919 } } }
I20260923 22:16:21.533082 20229 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:21.533200 20233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:21.534013 20233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:21.534919 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.528818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.1.132.190:33919)
0923 22:16:21.529012 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.529020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.529045 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:16:21.529478 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.529484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.529542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.529652 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.529898 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.531362 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.532388 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.533106 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.533561 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.533955 (+ 394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.533959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.533959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.534187 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.534232 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.534318 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.534639 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.534670 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.534750 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260923 22:16:21.535627 20233 catalog_manager.cc:1347] Generated new cluster ID: 0c9cd83510794023ad5c9f000ed821b0
I20260923 22:16:21.535676 20233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:21.562552 20233 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:21.563133 20233 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:21.570029 20233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: Generated new TSK 0
I20260923 22:16:21.570204 20233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:21.577016 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--builtin_ntp_servers=127.1.132.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:21.658572 20246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.658805 20246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.658844 20246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.660069 20246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.660125 20246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.660159 20246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:21.661677 20246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.662062 20246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.662523 20246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:21.668071 20246 server_base.cc:1034] running on GCE node
W20260923 22:16:21.667999 20251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.668058 20252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.668001 20254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.668506 20246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.668782 20246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.669950 20246 hybrid_clock.cc:648] HybridClock initialized: now 1790201781669936 us; error 29 us; skew 500 ppm
I20260923 22:16:21.671270 20246 webserver.cc:460] Webserver started at http://127.1.132.129:34771/ using document root <none> and password file <none>
I20260923 22:16:21.671496 20246 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.671546 20246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.671670 20246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.672865 20246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "787e98cea10c49a39881a27976eee753"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "f6c719784566b16919676606522b2097"
server_key_iv: "9ff7e049323364bbe59d51d3402fcf31"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.673192 20246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "787e98cea10c49a39881a27976eee753"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "f6c719784566b16919676606522b2097"
server_key_iv: "9ff7e049323364bbe59d51d3402fcf31"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.674427 20246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:16:21.675304 20260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.675519 20246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:21.675611 20246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "787e98cea10c49a39881a27976eee753"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "f6c719784566b16919676606522b2097"
server_key_iv: "9ff7e049323364bbe59d51d3402fcf31"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.675675 20246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.691507 20246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.691823 20246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.691956 20246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.692325 20246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:21.692728 20246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:21.692790 20246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.692844 20246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:21.692869 20246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.699025 20246 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:43927
I20260923 22:16:21.699069 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:43927 every 8 connection(s)
I20260923 22:16:21.699427 20246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:21.702371 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20246
I20260923 22:16:21.702455 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:21.702621 1554 external_mini_cluster.cc:1428] Setting key dced33526f4c9b43334d4c2c78010abd
I20260923 22:16:21.703619 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--builtin_ntp_servers=127.1.132.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:16:21.705050 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.700225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:46529 (local address 127.1.132.190:33919)
0923 22:16:21.700281 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.700283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.700491 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:16:21.701000 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.701001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.701018 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.701073 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.701738 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.702265 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.703583 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.703850 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.703961 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.704776 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.704777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.704778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.704782 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.704798 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.704802 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.704849 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.704944 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.704973 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:16:21.705266 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.700154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33919 (local address 127.1.132.129:46529)
0923 22:16:21.700375 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.700409 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.700905 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.701136 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.701140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.701150 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:21.701645 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.701649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.702362 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.702363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.703491 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.703494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.703975 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.703976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.704087 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.704637 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:21.704638 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:21.704667 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:21.704691 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:21.704860 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:21.704865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:21.704868 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.704941 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.705046 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260923 22:16:21.705736 20374 heartbeater.cc:347] Connected to a master server at 127.1.132.190:33919
I20260923 22:16:21.705823 20374 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:21.706000 20374 heartbeater.cc:511] Master 127.1.132.190:33919 requested a full tablet report, sending...
I20260923 22:16:21.706530 20186 ts_manager.cc:194] Registered new tserver with Master: 787e98cea10c49a39881a27976eee753 (127.1.132.129:43927)
I20260923 22:16:21.707468 20186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.129:46529
W20260923 22:16:21.790442 20377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.790679 20377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.790704 20377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.791951 20377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.791992 20377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.792019 20377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:21.793506 20377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.793745 20377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.794025 20377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:21.799443 20382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.799578 20377 server_base.cc:1034] running on GCE node
W20260923 22:16:21.799647 20383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.799443 20385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.799924 20377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.800153 20377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.801352 20377 hybrid_clock.cc:648] HybridClock initialized: now 1790201781801334 us; error 28 us; skew 500 ppm
I20260923 22:16:21.802729 20377 webserver.cc:460] Webserver started at http://127.1.132.130:35469/ using document root <none> and password file <none>
I20260923 22:16:21.802963 20377 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.803014 20377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.803136 20377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.804248 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "03018472b7564fa989be99fe000fa161"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a1a47a5c222d2231e7fb3d6ce628e613"
server_key_iv: "c1ebd80c53539af2a0c60cbb19a5044b"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.804600 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "03018472b7564fa989be99fe000fa161"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a1a47a5c222d2231e7fb3d6ce628e613"
server_key_iv: "c1ebd80c53539af2a0c60cbb19a5044b"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.806031 20377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:21.806788 20391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.806964 20377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:21.807060 20377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "03018472b7564fa989be99fe000fa161"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "a1a47a5c222d2231e7fb3d6ce628e613"
server_key_iv: "c1ebd80c53539af2a0c60cbb19a5044b"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.807125 20377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.825599 20377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.825980 20377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.826124 20377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.826367 20377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:21.826706 20377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:21.826740 20377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.826776 20377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:21.826799 20377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.832406 20377 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:46019
I20260923 22:16:21.832509 20504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:46019 every 8 connection(s)
I20260923 22:16:21.832849 20377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:21.838138 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.833544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:52891 (local address 127.1.132.190:33919)
0923 22:16:21.833608 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.833610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.833891 (+ 281us) server_negotiation.cc:407] Connection header received
0923 22:16:21.834358 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.834359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.834372 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.834435 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.835125 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.835618 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.836645 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.836878 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.836928 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.837904 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.837905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.837906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.837911 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.837927 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.837932 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.837976 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.838049 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.838074 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:16:21.838372 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.833542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33919 (local address 127.1.132.130:52891)
0923 22:16:21.833745 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.833789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.834261 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.834500 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.834504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.834513 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:21.835022 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.835027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.835708 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.835709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.836561 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.836564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.836938 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.836939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.837049 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.837604 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:21.837605 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:21.837636 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:21.837765 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:21.837984 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:21.837988 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:21.837991 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.838049 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.838143 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260923 22:16:21.838863 20505 heartbeater.cc:347] Connected to a master server at 127.1.132.190:33919
I20260923 22:16:21.838971 20505 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:21.839151 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20377
I20260923 22:16:21.839184 20505 heartbeater.cc:511] Master 127.1.132.190:33919 requested a full tablet report, sending...
I20260923 22:16:21.839257 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:21.839448 1554 external_mini_cluster.cc:1428] Setting key 8b8e50760807081bcdd11746cc02cc39
I20260923 22:16:21.839576 20186 ts_manager.cc:194] Registered new tserver with Master: 03018472b7564fa989be99fe000fa161 (127.1.132.130:46019)
I20260923 22:16:21.840090 20186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.130:52891
I20260923 22:16:21.840510 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--builtin_ntp_servers=127.1.132.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:21.922237 20508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:21.922454 20508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:21.922477 20508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:21.923606 20508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:21.923643 20508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:21.923667 20508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:21.925021 20508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:21.925230 20508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:21.925490 20508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:21.930481 20508 server_base.cc:1034] running on GCE node
W20260923 22:16:21.930431 20513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.930431 20516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:21.930429 20514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:21.930940 20508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:21.931159 20508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:21.932309 20508 hybrid_clock.cc:648] HybridClock initialized: now 1790201781932283 us; error 42 us; skew 500 ppm
I20260923 22:16:21.933669 20508 webserver.cc:460] Webserver started at http://127.1.132.131:40213/ using document root <none> and password file <none>
I20260923 22:16:21.933923 20508 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:21.933974 20508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:21.934103 20508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:21.935164 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "b9f8a16cf1c5442f9601fa07df391928"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "d34bb9be0ecd5077e336371cfa1b106f"
server_key_iv: "6ad992bfea1a97651b372b8ccac67433"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.935506 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "b9f8a16cf1c5442f9601fa07df391928"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "d34bb9be0ecd5077e336371cfa1b106f"
server_key_iv: "6ad992bfea1a97651b372b8ccac67433"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.936791 20508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:21.937453 20522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.937599 20508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:21.937680 20508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "b9f8a16cf1c5442f9601fa07df391928"
format_stamp: "Formatted at 2026-09-23 22:16:21 on dist-test-slave-l4q5"
server_key: "d34bb9be0ecd5077e336371cfa1b106f"
server_key_iv: "6ad992bfea1a97651b372b8ccac67433"
server_key_version: "encryptionkey@0"
I20260923 22:16:21.937739 20508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:21.947728 20508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:21.948029 20508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:21.948158 20508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:21.948411 20508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:21.948737 20508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:21.948769 20508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.948791 20508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:21.948813 20508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:21.954216 20508 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:41923
I20260923 22:16:21.954260 20635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:41923 every 8 connection(s)
I20260923 22:16:21.954586 20508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:21.955363 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20508
I20260923 22:16:21.955464 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:21.955646 1554 external_mini_cluster.cc:1428] Setting key f961939424e77a5dc91c1d36d0313a45
I20260923 22:16:21.959771 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.955285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:40543 (local address 127.1.132.190:33919)
0923 22:16:21.955357 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.955360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.955606 (+ 246us) server_negotiation.cc:407] Connection header received
0923 22:16:21.956261 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.956262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.956274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.956329 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.957028 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.957517 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.958478 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.958668 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.958723 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.959524 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.959525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.959526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.959531 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.959546 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.959550 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.959593 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.959665 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.959691 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:16:21.960002 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.955258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33919 (local address 127.1.132.131:40543)
0923 22:16:21.955464 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.955511 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.956140 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.956409 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.956414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.956424 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:21.956940 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.956944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.957603 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.957604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.958392 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.958394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.958741 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.958742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.958887 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.959388 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:21.959389 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:21.959419 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:21.959443 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:21.959598 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:21.959603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:21.959606 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.959669 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.959778 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260923 22:16:21.960453 20636 heartbeater.cc:347] Connected to a master server at 127.1.132.190:33919
I20260923 22:16:21.960549 20636 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:21.960726 20636 heartbeater.cc:511] Master 127.1.132.190:33919 requested a full tablet report, sending...
I20260923 22:16:21.961122 20186 ts_manager.cc:194] Registered new tserver with Master: b9f8a16cf1c5442f9601fa07df391928 (127.1.132.131:41923)
I20260923 22:16:21.961503 20186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.131:40543
I20260923 22:16:21.967818 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:21.972463 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.969253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.1.132.190:33919)
0923 22:16:21.969303 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.969306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.969414 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:16:21.969453 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.969453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.969463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.969512 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:21.969712 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.970276 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.971130 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.971326 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.971383 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.972135 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:21.972136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:21.972137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:21.972141 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:21.972157 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:21.972161 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:21.972299 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.972365 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.972377 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:21.974110 20186 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:21.975709 20186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:21.982177 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.978038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43927 (local address 127.0.0.1:50656)
0923 22:16:21.978334 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.978345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.978410 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.978691 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.978694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.978696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.978775 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.978778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.979842 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.979843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.981103 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.981105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.981873 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.981959 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.981973 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.982039 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.982095 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":4,"thread_start_us":112,"threads_started":1}
I20260923 22:16:21.982223 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.978146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.1.132.129:43927)
0923 22:16:21.978470 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.978475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.978484 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:21.978526 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.978528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.978545 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.978686 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.978853 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.979767 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981200 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.981791 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981858 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.981961 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.982053 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.982090 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":85,"threads_started":1}
I20260923 22:16:21.982442 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.978230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:41923 (local address 127.0.0.1:46764)
0923 22:16:21.978533 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.978542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.978579 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.978832 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.978833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.978835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.978867 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.978868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.980250 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.980251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.981048 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.981051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.982004 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.982005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.982068 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.982073 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.982121 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.982165 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":61,"threads_started":1}
I20260923 22:16:21.982524 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.977993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:46019 (local address 127.0.0.1:59822)
0923 22:16:21.978299 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:21.978339 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:21.978408 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:21.978695 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:21.978696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:21.978698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:21.978788 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.978790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.980212 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.980213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.981332 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:21.981334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.982299 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.982300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:21.982369 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.982372 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:21.982419 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:21.982455 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":33,"thread_start_us":93,"threads_started":1}
I20260923 22:16:21.982570 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.978445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.1.132.131:41923)
0923 22:16:21.978651 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.978655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.978663 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:21.978700 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.978701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.978715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.978791 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.978996 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.980109 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981135 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.981810 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981878 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.981966 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.982227 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.982426 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260923 22:16:21.982762 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:21.978118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.1.132.130:46019)
0923 22:16:21.978280 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:21.978285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:21.978491 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:16:21.978536 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:21.978538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:21.978553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:21.978622 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:21.978862 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.980098 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.981412 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:21.982128 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:21.982183 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:21.982266 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:21.982636 (+ 370us) server_negotiation.cc:299] Negotiation successful
0923 22:16:21.982684 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260923 22:16:21.983196 20308 tablet_service.cc:1467] Processing CreateTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 (DEFAULT_TABLE table=test-table [id=0eb515dd46214a27a8a66646f249e619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.983227 20570 tablet_service.cc:1467] Processing CreateTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 (DEFAULT_TABLE table=test-table [id=0eb515dd46214a27a8a66646f249e619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.983304 20439 tablet_service.cc:1467] Processing CreateTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 (DEFAULT_TABLE table=test-table [id=0eb515dd46214a27a8a66646f249e619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:21.983582 20308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cebfc1f1eb554199ba2cf7a54da25bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.983582 20570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cebfc1f1eb554199ba2cf7a54da25bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.983582 20439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cebfc1f1eb554199ba2cf7a54da25bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:21.986654 20650 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: Bootstrap starting.
I20260923 22:16:21.987047 20651 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Bootstrap starting.
I20260923 22:16:21.987562 20650 tablet_bootstrap.cc:654] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.987628 20651 tablet_bootstrap.cc:654] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.987924 20650 log.cc:824] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.987924 20651 log.cc:824] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.988293 20652 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Bootstrap starting.
I20260923 22:16:21.988660 20651 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: No bootstrap required, opened a new log
I20260923 22:16:21.988720 20650 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: No bootstrap required, opened a new log
I20260923 22:16:21.988722 20651 ts_tablet_manager.cc:1397] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:21.988781 20650 ts_tablet_manager.cc:1397] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:21.988914 20652 tablet_bootstrap.cc:654] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:21.989205 20652 log.cc:824] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:21.989965 20652 tablet_bootstrap.cc:492] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: No bootstrap required, opened a new log
I20260923 22:16:21.990031 20652 ts_tablet_manager.cc:1397] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:21.990767 20651 raft_consensus.cc:348] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.990935 20651 raft_consensus.cc:374] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.990978 20651 raft_consensus.cc:729] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03018472b7564fa989be99fe000fa161, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.991106 20651 consensus_queue.cc:260] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.991164 20650 raft_consensus.cc:348] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.991333 20650 raft_consensus.cc:374] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.991375 20650 raft_consensus.cc:729] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787e98cea10c49a39881a27976eee753, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.991380 20651 ts_tablet_manager.cc:1428] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:21.991503 20505 heartbeater.cc:503] Master 127.1.132.190:33919 was elected leader, sending a full tablet report...
I20260923 22:16:21.991482 20650 consensus_queue.cc:260] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.991724 20650 ts_tablet_manager.cc:1428] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:21.991775 20374 heartbeater.cc:503] Master 127.1.132.190:33919 was elected leader, sending a full tablet report...
I20260923 22:16:21.992430 20652 raft_consensus.cc:348] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.992568 20652 raft_consensus.cc:374] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:21.992610 20652 raft_consensus.cc:729] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f8a16cf1c5442f9601fa07df391928, State: Initialized, Role: FOLLOWER
I20260923 22:16:21.992743 20652 consensus_queue.cc:260] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:21.992964 20652 ts_tablet_manager.cc:1428] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:21.993036 20636 heartbeater.cc:503] Master 127.1.132.190:33919 was elected leader, sending a full tablet report...
W20260923 22:16:22.083488 20506 tablet.cc:2369] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:22.134879 20656 raft_consensus.cc:482] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:22.135006 20656 raft_consensus.cc:504] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:22.135380 20656 leader_election.cc:288] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 787e98cea10c49a39881a27976eee753 (127.1.132.129:43927), b9f8a16cf1c5442f9601fa07df391928 (127.1.132.131:41923)
I20260923 22:16:22.138198 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.135620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:44821 (local address 127.1.132.129:43927)
0923 22:16:22.135660 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:22.135661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:22.135672 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:22.135738 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:22.135739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:22.135751 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:22.135809 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:22.135977 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.136519 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.137513 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.137887 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.137941 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:22.137948 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:22.138091 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:16:22.138126 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:22.138186 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.135562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43927 (local address 127.1.132.130:44821)
0923 22:16:22.135629 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:22.135640 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:22.135683 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:22.135823 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:22.135825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:22.135827 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:22.135898 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:22.135901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.136589 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.136590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:22.137427 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:22.137430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.137959 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.137959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:22.138027 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:22.138031 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:22.138079 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:16:22.138114 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 22:16:22.138262 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.135632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:55963 (local address 127.1.132.131:41923)
0923 22:16:22.135737 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:22.135739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:22.135843 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:16:22.135908 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:22.135909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:22.135918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:22.135966 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:22.136128 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.136665 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.137581 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.137949 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.138017 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:22.138026 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:22.138166 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:16:22.138206 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:16:22.138317 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.135563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:41923 (local address 127.1.132.130:55963)
0923 22:16:22.135788 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:22.135796 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:22.135840 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:22.135974 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:22.135974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:22.135976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:22.136043 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:22.136045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.136755 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.136756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:22.137498 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:22.137501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.138026 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.138027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:22.138100 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:22.138103 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:22.138154 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:16:22.138192 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":59,"thread_start_us":96,"threads_started":1}
I20260923 22:16:22.138653 20590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cebfc1f1eb554199ba2cf7a54da25bb6" candidate_uuid: "03018472b7564fa989be99fe000fa161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8a16cf1c5442f9601fa07df391928" is_pre_election: true
I20260923 22:16:22.138811 20590 raft_consensus.cc:2454] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03018472b7564fa989be99fe000fa161 in term 0.
I20260923 22:16:22.138998 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cebfc1f1eb554199ba2cf7a54da25bb6" candidate_uuid: "03018472b7564fa989be99fe000fa161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787e98cea10c49a39881a27976eee753" is_pre_election: true
I20260923 22:16:22.139053 20394 leader_election.cc:302] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03018472b7564fa989be99fe000fa161, b9f8a16cf1c5442f9601fa07df391928; no voters:
I20260923 22:16:22.139144 20328 raft_consensus.cc:2454] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03018472b7564fa989be99fe000fa161 in term 0.
I20260923 22:16:22.139160 20656 raft_consensus.cc:2781] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:22.139230 20656 raft_consensus.cc:482] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:22.139250 20656 raft_consensus.cc:3037] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:22.139963 20656 raft_consensus.cc:504] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:22.140108 20656 leader_election.cc:288] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [CANDIDATE]: Term 1 election: Requested vote from peers 787e98cea10c49a39881a27976eee753 (127.1.132.129:43927), b9f8a16cf1c5442f9601fa07df391928 (127.1.132.131:41923)
I20260923 22:16:22.140252 20328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cebfc1f1eb554199ba2cf7a54da25bb6" candidate_uuid: "03018472b7564fa989be99fe000fa161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787e98cea10c49a39881a27976eee753"
I20260923 22:16:22.140283 20590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cebfc1f1eb554199ba2cf7a54da25bb6" candidate_uuid: "03018472b7564fa989be99fe000fa161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8a16cf1c5442f9601fa07df391928"
I20260923 22:16:22.140328 20328 raft_consensus.cc:3037] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:22.140343 20590 raft_consensus.cc:3037] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:22.141219 20590 raft_consensus.cc:2454] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03018472b7564fa989be99fe000fa161 in term 1.
I20260923 22:16:22.141232 20328 raft_consensus.cc:2454] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03018472b7564fa989be99fe000fa161 in term 1.
I20260923 22:16:22.141394 20393 leader_election.cc:302] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03018472b7564fa989be99fe000fa161, 787e98cea10c49a39881a27976eee753; no voters:
I20260923 22:16:22.141487 20656 raft_consensus.cc:2781] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:22.141635 20656 raft_consensus.cc:686] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 1 LEADER]: Becoming Leader. State: Replica: 03018472b7564fa989be99fe000fa161, State: Running, Role: LEADER
I20260923 22:16:22.141721 20656 consensus_queue.cc:237] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } }
I20260923 22:16:22.142568 20186 catalog_manager.cc:5495] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03018472b7564fa989be99fe000fa161 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "03018472b7564fa989be99fe000fa161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03018472b7564fa989be99fe000fa161" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 46019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:22.165166 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.162850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.1.132.130:46019)
0923 22:16:22.162897 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:22.162899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:22.162937 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:16:22.163016 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:22.163017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:22.163036 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:22.163085 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:22.163192 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.163655 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.164473 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:22.164655 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:22.164705 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:22.164884 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:22.164970 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:22.165037 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:22.165072 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:16:22.165079 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:22.166913 20590 raft_consensus.cc:1261] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 1 FOLLOWER]: Refusing update from remote peer 03018472b7564fa989be99fe000fa161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:22.166915 20328 raft_consensus.cc:1261] T cebfc1f1eb554199ba2cf7a54da25bb6 P 787e98cea10c49a39881a27976eee753 [term 1 FOLLOWER]: Refusing update from remote peer 03018472b7564fa989be99fe000fa161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:22.167295 20660 consensus_queue.cc:1035] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787e98cea10c49a39881a27976eee753" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:22.167434 20656 consensus_queue.cc:1035] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9f8a16cf1c5442f9601fa07df391928" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 41923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:22.171640 20665 mvcc.cc:205] Tried to move back new op lower bound from 7332666499753177088 to 7332666499653079040. Current Snapshot: MvccSnapshot[applied={T|T < 7332666499753177088 or (T in {7332666499753177088})}]
I20260923 22:16:22.172008 20185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:22.172106 20185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:22.172801 20185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9e363300e1f40bca9dcf0a5098ab3ff: Sending DeleteTablet for 3 replicas of tablet cebfc1f1eb554199ba2cf7a54da25bb6
I20260923 22:16:22.173147 20308 tablet_service.cc:1514] Processing DeleteTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:22 UTC) from {username='slave'} at 127.0.0.1:50656
I20260923 22:16:22.173183 20439 tablet_service.cc:1514] Processing DeleteTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:22 UTC) from {username='slave'} at 127.0.0.1:59822
I20260923 22:16:22.173417 20570 tablet_service.cc:1514] Processing DeleteTablet for tablet cebfc1f1eb554199ba2cf7a54da25bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:22 UTC) from {username='slave'} at 127.0.0.1:46764
I20260923 22:16:22.173573 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20246
I20260923 22:16:22.173606 20675 tablet_replica.cc:330] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: stopping tablet replica
I20260923 22:16:22.173692 20675 raft_consensus.cc:2229] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:22.173784 20675 raft_consensus.cc:2258] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:22.174353 20675 ts_tablet_manager.cc:1905] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:22.175925 20675 ts_tablet_manager.cc:1918] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:22.175994 20675 log.cc:1196] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/cebfc1f1eb554199ba2cf7a54da25bb6
I20260923 22:16:22.176239 20675 ts_tablet_manager.cc:1939] T cebfc1f1eb554199ba2cf7a54da25bb6 P b9f8a16cf1c5442f9601fa07df391928: Deleting consensus metadata
I20260923 22:16:22.176668 20172 catalog_manager.cc:4841] TS b9f8a16cf1c5442f9601fa07df391928 (127.1.132.131:41923): tablet cebfc1f1eb554199ba2cf7a54da25bb6 (table test-table [id=0eb515dd46214a27a8a66646f249e619]) successfully deleted
I20260923 22:16:22.180363 20674 tablet_replica.cc:330] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: stopping tablet replica
I20260923 22:16:22.180509 20674 raft_consensus.cc:2229] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:22.180631 20674 raft_consensus.cc:2258] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:22.181074 20674 ts_tablet_manager.cc:1905] T cebfc1f1eb554199ba2cf7a54da25bb6 P 03018472b7564fa989be99fe000fa161: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:16:22.181169 20171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:43927 (error 108)
I20260923 22:16:22.181661 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.181531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:43927 (local address 127.0.0.1:50660)
0923 22:16:22.181583 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:22.181627 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:16:22.181768 20171 catalog_manager.cc:4584] TS 787e98cea10c49a39881a27976eee753 (127.1.132.129:43927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cebfc1f1eb554199ba2cf7a54da25bb6: Network error: Client connection negotiation failed: client connection to 127.1.132.129:43927: connect: Connection refused (error 111)
I20260923 22:16:22.181995 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20377
I20260923 22:16:22.187202 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20508
I20260923 22:16:22.187235 20645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:22.187131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:46019 (local address 127.0.0.1:59834)
0923 22:16:22.187177 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:22.187202 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.130:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:16:22.191027 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20155
2026-09-23T22:16:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (824 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:22 dist-test-slave-l4q5 krb5kdc[20679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:22 dist-test-slave-l4q5 krb5kdc[20679](info): set up 2 sockets
Sep 23 22:16:22 dist-test-slave-l4q5 krb5kdc[20679](info): commencing operation
krb5kdc: starting...
W20260923 22:16:23.151813 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:23.271435 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:44337
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:44337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:23.351645 20692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:23.351858 20692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:23.351881 20692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:23.353029 20692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:23.353071 20692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:23.353085 20692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:23.353096 20692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:23.353106 20692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:23.354574 20692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:44337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:44337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:23.354811 20692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:23.355017 20692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:23.359794 20700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.359812 20698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.360008 20697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.360132 20692 server_base.cc:1034] running on GCE node
I20260923 22:16:23.360397 20692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:23.360674 20692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:23.361805 20692 hybrid_clock.cc:648] HybridClock initialized: now 1790201783361796 us; error 27 us; skew 500 ppm
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:23.364243 20692 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:23.364709 20692 webserver.cc:460] Webserver started at http://127.1.132.190:33967/ using document root <none> and password file <none>
I20260923 22:16:23.364907 20692 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:23.364954 20692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:23.365072 20692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:23.366155 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "31ed29db01334dc1bdd4164102f51760"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "5db8c14cfdda6147d01e51579e38b577"
server_key_iv: "78bff8e6ec0a7dc44ef991d65fb8bf4e"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.366473 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "31ed29db01334dc1bdd4164102f51760"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "5db8c14cfdda6147d01e51579e38b577"
server_key_iv: "78bff8e6ec0a7dc44ef991d65fb8bf4e"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.367707 20692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:23.368446 20707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.368594 20692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:23.368675 20692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "31ed29db01334dc1bdd4164102f51760"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "5db8c14cfdda6147d01e51579e38b577"
server_key_iv: "78bff8e6ec0a7dc44ef991d65fb8bf4e"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.368734 20692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:23.384150 20692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:23.386355 20692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:23.386507 20692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:23.390479 20692 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:44337
I20260923 22:16:23.390520 20759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:44337 every 8 connection(s)
I20260923 22:16:23.390878 20692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:23.391799 20760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:23.394178 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: Bootstrap starting.
I20260923 22:16:23.394795 20760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:23.395064 20760 log.cc:824] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:23.395864 20760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: No bootstrap required, opened a new log
I20260923 22:16:23.396167 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20692
I20260923 22:16:23.396298 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:23.396469 1554 external_mini_cluster.cc:1428] Setting key 7792eb66d7f04b6dfa347b7db4129f5d
I20260923 22:16:23.398031 20760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } }
I20260923 22:16:23.398219 20760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:23.398255 20760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ed29db01334dc1bdd4164102f51760, State: Initialized, Role: FOLLOWER
I20260923 22:16:23.398388 20760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } }
I20260923 22:16:23.398465 20760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:23.398488 20760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:23.398535 20760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:23.399631 20760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } }
I20260923 22:16:23.399745 20760 leader_election.cc:302] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ed29db01334dc1bdd4164102f51760; no voters:
I20260923 22:16:23.399917 20760 leader_election.cc:288] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:23.399961 20765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:23.400113 20765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [term 1 LEADER]: Becoming Leader. State: Replica: 31ed29db01334dc1bdd4164102f51760, State: Running, Role: LEADER
I20260923 22:16:23.400163 20760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:23.400238 20765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } }
I20260923 22:16:23.400605 20766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31ed29db01334dc1bdd4164102f51760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } } }
I20260923 22:16:23.400678 20766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:23.400844 20767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ed29db01334dc1bdd4164102f51760. Latest consensus state: current_term: 1 leader_uuid: "31ed29db01334dc1bdd4164102f51760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ed29db01334dc1bdd4164102f51760" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 44337 } } }
I20260923 22:16:23.400895 20767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:23.400997 20772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:23.401477 20772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:23.403347 20772 catalog_manager.cc:1347] Generated new cluster ID: b7398b86ccf748af8e0ce4d423f5ccc0
I20260923 22:16:23.403395 20772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:23.408680 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.396978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.1.132.190:44337)
0923 22:16:23.397174 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.397184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.397218 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:16:23.397993 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.398003 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.398079 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.398241 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.398988 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.400395 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.401539 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.402269 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.402371 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.405210 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.405214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.405215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:23.405227 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.407063 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.407474 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.407477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.407478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.407603 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.407810 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.407811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.407811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.408084 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.408251 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.408362 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.408491 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260923 22:16:23.413087 20772 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:23.413614 20772 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:23.417048 20772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: Generated new TSK 0
I20260923 22:16:23.417186 20772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
Entry for principal kudu/127.1.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab.
I20260923 22:16:23.447603 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:23.538952 20786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:23.539180 20786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:23.539215 20786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:23.540462 20786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:23.540504 20786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:23.540541 20786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:23.542074 20786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.129.keytab
--principal=kudu/127.1.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:23.542331 20786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:23.542543 20786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:23.547508 20791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.547513 20794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.547588 20786 server_base.cc:1034] running on GCE node
W20260923 22:16:23.547513 20792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.547952 20786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:23.548380 20786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:23.549592 20786 hybrid_clock.cc:648] HybridClock initialized: now 1790201783549579 us; error 34 us; skew 500 ppm
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:23.552073 20786 init.cc:377] Logged in from keytab as kudu/127.1.132.129@KRBTEST.COM (short username kudu)
I20260923 22:16:23.552544 20786 webserver.cc:460] Webserver started at http://127.1.132.129:38013/ using document root <none> and password file <none>
I20260923 22:16:23.552726 20786 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:23.552764 20786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:23.552865 20786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:23.553916 20786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "31c0ca7e7a624c3692aa07d4bd24dc44"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2934458fd8139b790629ad45127dbdca"
server_key_iv: "953cdd144a55b61ecd7ad57cb84a8322"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.554208 20786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "31c0ca7e7a624c3692aa07d4bd24dc44"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2934458fd8139b790629ad45127dbdca"
server_key_iv: "953cdd144a55b61ecd7ad57cb84a8322"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.555402 20786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:23.556311 20801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.556501 20786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:23.556597 20786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "31c0ca7e7a624c3692aa07d4bd24dc44"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2934458fd8139b790629ad45127dbdca"
server_key_iv: "953cdd144a55b61ecd7ad57cb84a8322"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.556684 20786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:23.574518 20786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:23.576478 20786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:23.576613 20786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:23.576853 20786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:23.577153 20786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:23.577186 20786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.577215 20786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:23.577234 20786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.582742 20786 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:37639
I20260923 22:16:23.582834 20914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:37639 every 8 connection(s)
I20260923 22:16:23.583102 20786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.129@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:23.592783 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.583875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45273 (local address 127.1.132.190:44337)
0923 22:16:23.583956 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.583957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.584102 (+ 145us) server_negotiation.cc:407] Connection header received
0923 22:16:23.584606 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.584607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.584618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.584678 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.585641 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.586226 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.587300 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.587515 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.587580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.589977 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.589984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.589984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:23.589991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.591279 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.591727 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.591728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.591729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.591792 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.592099 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.592100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.592100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.592268 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.592333 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.592594 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.592685 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 22:16:23.593076 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20786
I20260923 22:16:23.593173 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:23.593351 1554 external_mini_cluster.cc:1428] Setting key 031e6fa5f239b1532c03876f385797e0
I20260923 22:16:23.593441 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.583828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44337 (local address 127.1.132.129:45273)
0923 22:16:23.583985 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.584026 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.584527 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.584751 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.584755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.585327 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:23.585530 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.585535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.586340 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.586341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.587201 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.587203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.587622 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.587623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.587756 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.588372 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:23.588381 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:23.589868 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:23.591367 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.591374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.591376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.591603 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.591880 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.591881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.591882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.592011 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.592326 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:23.592330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:23.592456 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.592541 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.592674 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260923 22:16:23.593945 20915 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44337
I20260923 22:16:23.594049 20915 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:23.594249 20915 heartbeater.cc:511] Master 127.1.132.190:44337 requested a full tablet report, sending...
I20260923 22:16:23.594817 20723 ts_manager.cc:194] Registered new tserver with Master: 31c0ca7e7a624c3692aa07d4bd24dc44 (127.1.132.129:37639)
I20260923 22:16:23.595440 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.129@KRBTEST.COM'} at 127.1.132.129:45273
WARNING: no policy specified for kudu/127.1.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
Entry for principal kudu/127.1.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab.
I20260923 22:16:23.619328 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:23.697690 20920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:23.697898 20920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:23.697919 20920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:23.699038 20920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:23.699077 20920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:23.699101 20920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:23.700560 20920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.130.keytab
--principal=kudu/127.1.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:23.700781 20920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:23.700994 20920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:23.705685 20928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.705783 20920 server_base.cc:1034] running on GCE node
W20260923 22:16:23.705691 20925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.705685 20926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.706033 20920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:23.706251 20920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:23.707391 20920 hybrid_clock.cc:648] HybridClock initialized: now 1790201783707378 us; error 27 us; skew 500 ppm
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:23.709818 20920 init.cc:377] Logged in from keytab as kudu/127.1.132.130@KRBTEST.COM (short username kudu)
I20260923 22:16:23.710304 20920 webserver.cc:460] Webserver started at http://127.1.132.130:45967/ using document root <none> and password file <none>
I20260923 22:16:23.710496 20920 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:23.710536 20920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:23.710623 20920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:23.711637 20920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "4312f42dd4eb4a1f8bda2de61d17650e"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "ac34e045074dfdaf5e639d2ab7303116"
server_key_iv: "8110b0517e7c26a2f4f958a5d4a77f32"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.711927 20920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "4312f42dd4eb4a1f8bda2de61d17650e"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "ac34e045074dfdaf5e639d2ab7303116"
server_key_iv: "8110b0517e7c26a2f4f958a5d4a77f32"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.713125 20920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:23.713804 20935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.713954 20920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:23.714046 20920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "4312f42dd4eb4a1f8bda2de61d17650e"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "ac34e045074dfdaf5e639d2ab7303116"
server_key_iv: "8110b0517e7c26a2f4f958a5d4a77f32"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.714107 20920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:23.740751 20920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:23.742919 20920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:23.743065 20920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:23.743297 20920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:23.743602 20920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:23.743634 20920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.743665 20920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:23.743688 20920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.749486 20920 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:44531
I20260923 22:16:23.749526 21048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:44531 every 8 connection(s)
I20260923 22:16:23.749840 20920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:23.754083 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 20920
I20260923 22:16:23.754186 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:23.754357 1554 external_mini_cluster.cc:1428] Setting key 861eca6f2d67d7857449b7009d1a1b3c
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.130@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:23.759467 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.750517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:46673 (local address 127.1.132.190:44337)
0923 22:16:23.750576 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.750578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.750776 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:16:23.751305 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.751315 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.751328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.751382 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.752367 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.752866 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.753822 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.754015 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.754073 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.756472 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.756473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.756474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:23.756482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.757908 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.758329 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.758331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.758331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.758395 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.758700 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.758701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.758702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.758848 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.758921 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.759260 (+ 339us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.759363 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:16:23.759634 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.750469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44337 (local address 127.1.132.130:46673)
0923 22:16:23.750645 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.750695 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.751210 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.751461 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.751465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.752027 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:23.752265 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.752271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.752949 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.752950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.753738 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.753740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.754084 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.754085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.754218 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.754777 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:23.754786 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:23.756375 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:23.757998 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.758005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.758007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.758233 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.758480 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.758481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.758481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.758618 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.758934 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:23.758940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:23.759084 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.759223 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.759425 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260923 22:16:23.760231 21049 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44337
I20260923 22:16:23.760349 21049 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:23.760551 21049 heartbeater.cc:511] Master 127.1.132.190:44337 requested a full tablet report, sending...
I20260923 22:16:23.760944 20723 ts_manager.cc:194] Registered new tserver with Master: 4312f42dd4eb4a1f8bda2de61d17650e (127.1.132.130:44531)
I20260923 22:16:23.761317 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.130@KRBTEST.COM'} at 127.1.132.130:46673
WARNING: no policy specified for kudu/127.1.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
Entry for principal kudu/127.1.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab.
I20260923 22:16:23.781159 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:23.861663 21054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:23.861883 21054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:23.861915 21054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:23.863119 21054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:23.863169 21054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:23.863205 21054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:23.864813 21054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.131.keytab
--principal=kudu/127.1.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:44337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:23.865170 21054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:23.865414 21054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:23.870344 21059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.870316 21062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:23.870460 21060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:23.870497 21054 server_base.cc:1034] running on GCE node
I20260923 22:16:23.870672 21054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:23.870961 21054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:23.872120 21054 hybrid_clock.cc:648] HybridClock initialized: now 1790201783872098 us; error 37 us; skew 500 ppm
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:23.874567 21054 init.cc:377] Logged in from keytab as kudu/127.1.132.131@KRBTEST.COM (short username kudu)
I20260923 22:16:23.875006 21054 webserver.cc:460] Webserver started at http://127.1.132.131:43175/ using document root <none> and password file <none>
I20260923 22:16:23.875213 21054 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:23.875262 21054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:23.875385 21054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:23.876508 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "2ea47bc057574e6380b35547cff27412"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2326b046516d8aa80fdc422500ff2c3a"
server_key_iv: "adfbbf930f991906f7f7347ea4d643e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.876858 21054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea47bc057574e6380b35547cff27412"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2326b046516d8aa80fdc422500ff2c3a"
server_key_iv: "adfbbf930f991906f7f7347ea4d643e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.878050 21054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:23.878801 21069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.878966 21054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:23.879050 21054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "2ea47bc057574e6380b35547cff27412"
format_stamp: "Formatted at 2026-09-23 22:16:23 on dist-test-slave-l4q5"
server_key: "2326b046516d8aa80fdc422500ff2c3a"
server_key_iv: "adfbbf930f991906f7f7347ea4d643e8"
server_key_version: "encryptionkey@0"
I20260923 22:16:23.879112 21054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:23.894057 21054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:23.896453 21054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:23.896589 21054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:23.896817 21054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:23.897122 21054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:23.897161 21054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.897189 21054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:23.897231 21054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:23.903121 21054 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:35629
I20260923 22:16:23.903177 21182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:35629 every 8 connection(s)
I20260923 22:16:23.903478 21054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:23.906087 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21054
I20260923 22:16:23.906191 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:23.906369 1554 external_mini_cluster.cc:1428] Setting key 090c9a6c7b47a08225f6680f2ad50610
Sep 23 22:16:23 dist-test-slave-l4q5 krb5kdc[20679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.131@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:23.912649 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.904206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:42075 (local address 127.1.132.190:44337)
0923 22:16:23.904249 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.904251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.904487 (+ 236us) server_negotiation.cc:407] Connection header received
0923 22:16:23.904990 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.904991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.905003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.905066 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.906214 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.906825 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.907863 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.908066 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.908120 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.910291 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.910292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.910293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:23.910299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.911454 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.911802 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.911803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.911803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.911846 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.912062 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.912063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.912063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.912215 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.912280 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.912519 (+ 239us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.912583 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:23.912827 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.904143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:44337 (local address 127.1.132.131:42075)
0923 22:16:23.904333 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.904377 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.904891 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.905171 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.905175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.905749 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:23.906075 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.906082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.906909 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.906909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.907789 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.907792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.908127 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.908128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.908276 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.908808 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:23.908817 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:23.910192 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:23.911521 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.911527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.911529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.911745 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.911889 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:23.911890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:23.911890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:23.911980 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:23.912273 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:23.912276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:23.912406 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.912490 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.912614 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260923 22:16:23.913264 21183 heartbeater.cc:347] Connected to a master server at 127.1.132.190:44337
I20260923 22:16:23.913357 21183 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:23.913530 21183 heartbeater.cc:511] Master 127.1.132.190:44337 requested a full tablet report, sending...
I20260923 22:16:23.913908 20723 ts_manager.cc:194] Registered new tserver with Master: 2ea47bc057574e6380b35547cff27412 (127.1.132.131:35629)
I20260923 22:16:23.914238 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.132.131@KRBTEST.COM'} at 127.1.132.131:42075
I20260923 22:16:23.918079 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:23.924696 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.919315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.1.132.190:44337)
0923 22:16:23.919380 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.919383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.919456 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:16:23.919520 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.919521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.919530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.919572 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:23.920092 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.920623 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.921307 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.921509 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.921562 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.922619 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:23.922620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:23.922621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:23.922627 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:23.923666 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.923960 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.923961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.923961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.924016 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:23.924168 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:23.924169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:23.924170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:23.924345 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:23.924434 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.924532 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.924609 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:16:23.926070 20723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:23.927577 20723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:23.933943 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.929923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:44531 (local address 127.0.0.1:38108)
0923 22:16:23.930350 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.930359 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.930410 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.930639 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.930640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.930642 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:23.930694 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.930696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931903 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.931904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.932851 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.932853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.933684 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.933739 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.933751 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.933811 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.933859 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":76,"threads_started":1}
I20260923 22:16:23.933943 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.930168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.1.132.130:44531)
0923 22:16:23.930355 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.930360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.930401 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:16:23.930478 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.930481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.930495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.930564 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:23.930767 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931821 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.932921 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.933611 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933685 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.933744 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.933826 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.933866 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260923 22:16:23.934053 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.929742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:35629 (local address 127.0.0.1:58078)
0923 22:16:23.930116 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.930126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.930178 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.930362 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.930365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.930368 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:23.930459 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.930461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931741 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.931742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.932974 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.932977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.933852 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.933916 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.933920 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.933967 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.934001 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":40,"thread_start_us":75,"threads_started":1}
I20260923 22:16:23.934252 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.929945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.1.132.131:35629)
0923 22:16:23.930114 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.930118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.930172 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:16:23.930234 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.930236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.930251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.930333 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:23.930544 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931647 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933060 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.933753 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933815 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.933864 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.934141 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.934180 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260923 22:16:23.934909 20983 tablet_service.cc:1467] Processing CreateTablet for tablet 65d967a84fb747f4a129802109eff881 (DEFAULT_TABLE table=test-table [id=31b4344efe964021a21e3442c94d1f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:23.934940 21117 tablet_service.cc:1467] Processing CreateTablet for tablet 65d967a84fb747f4a129802109eff881 (DEFAULT_TABLE table=test-table [id=31b4344efe964021a21e3442c94d1f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:23.935072 21194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.929741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37639 (local address 127.0.0.1:48380)
0923 22:16:23.929835 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:23.929873 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:23.929930 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:23.930687 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:23.930687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:23.930689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:23.930722 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.930723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931587 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.931588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.932975 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:23.932977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.934852 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.934853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:23.934914 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.934918 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:23.934973 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:16:23.935012 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":62,"thread_start_us":37,"threads_started":1}
I20260923 22:16:23.935113 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:23.929957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.1.132.129:37639)
0923 22:16:23.930532 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:23.930536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:23.930545 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:23.930580 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:23.930582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:23.930596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:23.930788 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:23.930849 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.931501 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.933900 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:23.934755 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:23.934824 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:23.934877 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:23.934998 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 22:16:23.935040 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":78,"threads_started":1}
I20260923 22:16:23.935273 21117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d967a84fb747f4a129802109eff881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:23.935346 20983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d967a84fb747f4a129802109eff881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:23.935673 20849 tablet_service.cc:1467] Processing CreateTablet for tablet 65d967a84fb747f4a129802109eff881 (DEFAULT_TABLE table=test-table [id=31b4344efe964021a21e3442c94d1f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:23.935948 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d967a84fb747f4a129802109eff881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:23.938068 21200 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Bootstrap starting.
I20260923 22:16:23.938504 21201 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Bootstrap starting.
I20260923 22:16:23.938666 21200 tablet_bootstrap.cc:654] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:23.939121 21201 tablet_bootstrap.cc:654] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:23.939203 21200 log.cc:824] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:23.939229 21202 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Bootstrap starting.
I20260923 22:16:23.939613 21201 log.cc:824] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:23.939822 21202 tablet_bootstrap.cc:654] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:23.940009 21200 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: No bootstrap required, opened a new log
I20260923 22:16:23.940078 21200 ts_tablet_manager.cc:1397] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:23.940349 21201 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: No bootstrap required, opened a new log
I20260923 22:16:23.940368 21202 log.cc:824] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:23.940414 21201 ts_tablet_manager.cc:1397] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:23.940934 21202 tablet_bootstrap.cc:492] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: No bootstrap required, opened a new log
I20260923 22:16:23.940995 21202 ts_tablet_manager.cc:1397] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:23.941990 21200 raft_consensus.cc:348] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.942170 21200 raft_consensus.cc:374] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:23.942209 21200 raft_consensus.cc:729] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4312f42dd4eb4a1f8bda2de61d17650e, State: Initialized, Role: FOLLOWER
I20260923 22:16:23.942200 21201 raft_consensus.cc:348] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.942349 21201 raft_consensus.cc:374] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:23.942391 21201 raft_consensus.cc:729] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47bc057574e6380b35547cff27412, State: Initialized, Role: FOLLOWER
I20260923 22:16:23.942363 21200 consensus_queue.cc:260] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.942497 21201 consensus_queue.cc:260] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.942627 21200 ts_tablet_manager.cc:1428] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:23.942556 21202 raft_consensus.cc:348] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.942688 21049 heartbeater.cc:503] Master 127.1.132.190:44337 was elected leader, sending a full tablet report...
I20260923 22:16:23.942720 21202 raft_consensus.cc:374] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:23.942763 21202 raft_consensus.cc:729] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c0ca7e7a624c3692aa07d4bd24dc44, State: Initialized, Role: FOLLOWER
I20260923 22:16:23.942803 21183 heartbeater.cc:503] Master 127.1.132.190:44337 was elected leader, sending a full tablet report...
I20260923 22:16:23.942874 21202 consensus_queue.cc:260] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:23.943050 21201 ts_tablet_manager.cc:1428] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:16:23.943123 21202 ts_tablet_manager.cc:1428] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:16:23.943228 20915 heartbeater.cc:503] Master 127.1.132.190:44337 was elected leader, sending a full tablet report...
W20260923 22:16:24.000380 21050 tablet.cc:2369] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:16:24.083807 20916 tablet.cc:2369] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:24.098806 21208 raft_consensus.cc:482] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:24.098966 21208 raft_consensus.cc:504] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:24.099308 21208 leader_election.cc:288] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4312f42dd4eb4a1f8bda2de61d17650e (127.1.132.130:44531), 2ea47bc057574e6380b35547cff27412 (127.1.132.131:35629)
I20260923 22:16:24.102739 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.099624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45681 (local address 127.1.132.131:35629)
0923 22:16:24.099669 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.099671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.099680 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:24.099786 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.099787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.099812 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.099876 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.100103 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.100810 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.101916 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.102393 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.102447 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.102453 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.102632 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.102679 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:16:24.102754 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.099500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:35629 (local address 127.1.132.129:45681)
0923 22:16:24.099602 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.099614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.099675 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.099900 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.099901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.099904 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.099984 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.099986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.100884 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.100885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.101851 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.101854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.102465 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.102466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.102559 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.102564 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.102620 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.102660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260923 22:16:24.102941 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.099934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:58725 (local address 127.1.132.130:44531)
0923 22:16:24.100158 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.100160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.100168 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:24.100225 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.100226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.100236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.100290 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.100464 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.101001 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.102166 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.102601 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.102658 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.102665 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.102836 (+ 171us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.102880 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260923 22:16:24.103135 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.099519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:44531 (local address 127.1.132.129:58725)
0923 22:16:24.099990 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.100000 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.100050 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.100316 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.100318 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.100320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.100393 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.100395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.101067 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.101067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.102078 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.102081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.102695 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.102696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.102759 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.102762 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.103025 (+ 263us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.103064 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":94,"threads_started":1}
I20260923 22:16:24.103093 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d967a84fb747f4a129802109eff881" candidate_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47bc057574e6380b35547cff27412" is_pre_election: true
I20260923 22:16:24.103255 21137 raft_consensus.cc:2454] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c0ca7e7a624c3692aa07d4bd24dc44 in term 0.
I20260923 22:16:24.103456 20802 leader_election.cc:302] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47bc057574e6380b35547cff27412, 31c0ca7e7a624c3692aa07d4bd24dc44; no voters:
I20260923 22:16:24.103434 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d967a84fb747f4a129802109eff881" candidate_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" is_pre_election: true
I20260923 22:16:24.103564 21003 raft_consensus.cc:2454] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c0ca7e7a624c3692aa07d4bd24dc44 in term 0.
I20260923 22:16:24.103652 21208 raft_consensus.cc:2781] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:24.103729 21208 raft_consensus.cc:482] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:24.103752 21208 raft_consensus.cc:3037] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.104480 21208 raft_consensus.cc:504] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:24.104597 21208 leader_election.cc:288] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [CANDIDATE]: Term 1 election: Requested vote from peers 4312f42dd4eb4a1f8bda2de61d17650e (127.1.132.130:44531), 2ea47bc057574e6380b35547cff27412 (127.1.132.131:35629)
I20260923 22:16:24.104753 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d967a84fb747f4a129802109eff881" candidate_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47bc057574e6380b35547cff27412"
I20260923 22:16:24.104775 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d967a84fb747f4a129802109eff881" candidate_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4312f42dd4eb4a1f8bda2de61d17650e"
I20260923 22:16:24.104827 21137 raft_consensus.cc:3037] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.104833 21003 raft_consensus.cc:3037] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.105634 21003 raft_consensus.cc:2454] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c0ca7e7a624c3692aa07d4bd24dc44 in term 1.
I20260923 22:16:24.105643 21137 raft_consensus.cc:2454] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c0ca7e7a624c3692aa07d4bd24dc44 in term 1.
I20260923 22:16:24.105793 20802 leader_election.cc:302] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47bc057574e6380b35547cff27412, 31c0ca7e7a624c3692aa07d4bd24dc44; no voters:
I20260923 22:16:24.105894 21208 raft_consensus.cc:2781] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:24.106070 21208 raft_consensus.cc:686] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [term 1 LEADER]: Becoming Leader. State: Replica: 31c0ca7e7a624c3692aa07d4bd24dc44, State: Running, Role: LEADER
I20260923 22:16:24.106160 21208 consensus_queue.cc:237] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } }
I20260923 22:16:24.106927 20724 catalog_manager.cc:5495] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31c0ca7e7a624c3692aa07d4bd24dc44 (127.1.132.129). New cstate: current_term: 1 leader_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c0ca7e7a624c3692aa07d4bd24dc44" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 37639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:24.117843 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.115606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.1.132.129:37639)
0923 22:16:24.115662 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.115664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.115673 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:24.115729 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.115730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.115747 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.115805 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:24.115913 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.116413 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.117151 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.117366 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.117421 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.117555 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:24.117652 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:24.117728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.117750 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.117768 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:16:24.119560 21003 raft_consensus.cc:1261] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 1 FOLLOWER]: Refusing update from remote peer 31c0ca7e7a624c3692aa07d4bd24dc44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:24.119644 21137 raft_consensus.cc:1261] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 1 FOLLOWER]: Refusing update from remote peer 31c0ca7e7a624c3692aa07d4bd24dc44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:24.119868 21208 consensus_queue.cc:1035] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4312f42dd4eb4a1f8bda2de61d17650e" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 44531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:24.119993 21211 consensus_queue.cc:1035] T 65d967a84fb747f4a129802109eff881 P 31c0ca7e7a624c3692aa07d4bd24dc44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea47bc057574e6380b35547cff27412" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 35629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:24.122174 21214 mvcc.cc:205] Tried to move back new op lower bound from 7332666507751182336 to 7332666507699449856. Current Snapshot: MvccSnapshot[applied={T|T < 7332666507751182336}]
I20260923 22:16:24.124322 20723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:24.124409 20723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:24.125077 20723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31ed29db01334dc1bdd4164102f51760: Sending DeleteTablet for 3 replicas of tablet 65d967a84fb747f4a129802109eff881
I20260923 22:16:24.125372 21117 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d967a84fb747f4a129802109eff881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:58078
I20260923 22:16:24.125408 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d967a84fb747f4a129802109eff881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:48380
I20260923 22:16:24.125579 20983 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d967a84fb747f4a129802109eff881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='kudu', principal='kudu/127.1.132.190@KRBTEST.COM'} at 127.0.0.1:38108
I20260923 22:16:24.125800 21224 tablet_replica.cc:330] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: stopping tablet replica
I20260923 22:16:24.125808 21226 tablet_replica.cc:330] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: stopping tablet replica
I20260923 22:16:24.125891 21224 raft_consensus.cc:2229] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:24.125895 21226 raft_consensus.cc:2229] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:24.125983 21226 raft_consensus.cc:2258] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:24.125986 21224 raft_consensus.cc:2258] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:24.126025 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20786
I20260923 22:16:24.126894 21226 ts_tablet_manager.cc:1905] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:24.128504 21224 ts_tablet_manager.cc:1905] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:24.128518 21226 ts_tablet_manager.cc:1918] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:24.128575 21226 log.cc:1196] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-1/wal/wals/65d967a84fb747f4a129802109eff881
I20260923 22:16:24.128829 21226 ts_tablet_manager.cc:1939] T 65d967a84fb747f4a129802109eff881 P 4312f42dd4eb4a1f8bda2de61d17650e: Deleting consensus metadata
I20260923 22:16:24.129186 20709 catalog_manager.cc:4841] TS 4312f42dd4eb4a1f8bda2de61d17650e (127.1.132.130:44531): tablet 65d967a84fb747f4a129802109eff881 (table test-table [id=31b4344efe964021a21e3442c94d1f3e]) successfully deleted
I20260923 22:16:24.129808 21224 ts_tablet_manager.cc:1918] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:24.129866 21224 log.cc:1196] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/65d967a84fb747f4a129802109eff881
I20260923 22:16:24.130065 21224 ts_tablet_manager.cc:1939] T 65d967a84fb747f4a129802109eff881 P 2ea47bc057574e6380b35547cff27412: Deleting consensus metadata
I20260923 22:16:24.130584 20708 catalog_manager.cc:4841] TS 2ea47bc057574e6380b35547cff27412 (127.1.132.131:35629): tablet 65d967a84fb747f4a129802109eff881 (table test-table [id=31b4344efe964021a21e3442c94d1f3e]) successfully deleted
W20260923 22:16:24.131047 20709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:37639 (error 108)
I20260923 22:16:24.131259 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20920
I20260923 22:16:24.131492 21194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.131389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:37639 (local address 127.0.0.1:48392)
0923 22:16:24.131435 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.131462 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:16:24.131582 20709 catalog_manager.cc:4584] TS 31c0ca7e7a624c3692aa07d4bd24dc44 (127.1.132.129:37639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65d967a84fb747f4a129802109eff881: Network error: Client connection negotiation failed: client connection to 127.1.132.129:37639: connect: Connection refused (error 111)
I20260923 22:16:24.135830 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21054
I20260923 22:16:24.139605 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 20692
2026-09-23T22:16:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T22:16:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:24Z Disabled control of system clock
I20260923 22:16:24.165580 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42971
--webserver_interface=127.1.132.190
--webserver_port=0
--builtin_ntp_servers=127.1.132.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:42971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:24.243314 21231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:24.243516 21231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:24.243535 21231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:24.244630 21231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:24.244664 21231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:24.244675 21231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:24.244688 21231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:24.244697 21231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:24.245963 21231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:42971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:42971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:24.246140 21231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:24.246364 21231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:24.250816 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.250902 21231 server_base.cc:1034] running on GCE node
W20260923 22:16:24.250819 21239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.250981 21236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.251230 21231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:24.251497 21231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:24.252666 21231 hybrid_clock.cc:648] HybridClock initialized: now 1790201784252654 us; error 27 us; skew 500 ppm
I20260923 22:16:24.253912 21231 webserver.cc:460] Webserver started at http://127.1.132.190:35199/ using document root <none> and password file <none>
I20260923 22:16:24.254135 21231 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:24.254184 21231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:24.254314 21231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:24.255347 21231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "045dc4d0b9f2423baa833fb1c02742f6"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "8b0c508c0cdf355c20b5338c948f538b"
server_key_iv: "72791418524918a1d5acca0abf01fdda"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.255661 21231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "045dc4d0b9f2423baa833fb1c02742f6"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "8b0c508c0cdf355c20b5338c948f538b"
server_key_iv: "72791418524918a1d5acca0abf01fdda"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.256975 21231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:24.257705 21245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.257893 21231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:24.257980 21231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "045dc4d0b9f2423baa833fb1c02742f6"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "8b0c508c0cdf355c20b5338c948f538b"
server_key_iv: "72791418524918a1d5acca0abf01fdda"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.258041 21231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:24.279724 21231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:24.280037 21231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:24.280167 21231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:24.283805 21231 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:42971
I20260923 22:16:24.283866 21297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:42971 every 8 connection(s)
I20260923 22:16:24.284160 21231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:24.285097 21298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:24.287324 21298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: Bootstrap starting.
I20260923 22:16:24.287897 21298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:24.288161 21298 log.cc:824] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:24.288931 21298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: No bootstrap required, opened a new log
I20260923 22:16:24.289608 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21231
I20260923 22:16:24.289682 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:24.289865 1554 external_mini_cluster.cc:1428] Setting key a1267aa626f51f760a9f19a6bea579a1
I20260923 22:16:24.290905 21298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } }
I20260923 22:16:24.291083 21298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:24.291119 21298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045dc4d0b9f2423baa833fb1c02742f6, State: Initialized, Role: FOLLOWER
I20260923 22:16:24.291245 21298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } }
I20260923 22:16:24.291332 21298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:24.291361 21298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:24.291407 21298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.292145 21298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } }
I20260923 22:16:24.292276 21298 leader_election.cc:302] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 045dc4d0b9f2423baa833fb1c02742f6; no voters:
I20260923 22:16:24.292425 21298 leader_election.cc:288] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:24.292485 21303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:24.292640 21303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [term 1 LEADER]: Becoming Leader. State: Replica: 045dc4d0b9f2423baa833fb1c02742f6, State: Running, Role: LEADER
I20260923 22:16:24.292670 21298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:24.292732 21303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } }
I20260923 22:16:24.293195 21304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "045dc4d0b9f2423baa833fb1c02742f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } } }
I20260923 22:16:24.293210 21305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 045dc4d0b9f2423baa833fb1c02742f6. Latest consensus state: current_term: 1 leader_uuid: "045dc4d0b9f2423baa833fb1c02742f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045dc4d0b9f2423baa833fb1c02742f6" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 42971 } } }
I20260923 22:16:24.293290 21304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:24.293292 21305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:24.294082 21312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:24.294536 21312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:24.296097 21312 catalog_manager.cc:1347] Generated new cluster ID: 63ad1bdb357b46aca5378f9aba2474ba
I20260923 22:16:24.296149 21312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:24.300364 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.290386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.1.132.190:42971)
0923 22:16:24.290636 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.290646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.290685 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:16:24.290859 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.290867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.290940 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.291097 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.291324 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.293018 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.294139 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.294566 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.294967 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.295349 (+ 382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.295353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.295353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:24.295518 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.295556 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:24.295570 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.295631 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.295814 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.295892 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260923 22:16:24.331723 21312 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:24.332468 21312 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:24.339950 21312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: Generated new TSK 0
I20260923 22:16:24.340157 21312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:24.345633 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.129:0
--local_ip_for_outbound_sockets=127.1.132.129
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--builtin_ntp_servers=127.1.132.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:24.426388 21322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:24.426613 21322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:24.426645 21322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:24.427892 21322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:24.427938 21322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:24.427975 21322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.129
I20260923 22:16:24.429451 21322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.132.129
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.129
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:24.429723 21322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:24.430027 21322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:24.435492 21330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.435482 21328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.435482 21327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.435834 21322 server_base.cc:1034] running on GCE node
I20260923 22:16:24.436059 21322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:24.436336 21322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:24.437482 21322 hybrid_clock.cc:648] HybridClock initialized: now 1790201784437468 us; error 30 us; skew 500 ppm
I20260923 22:16:24.438773 21322 webserver.cc:460] Webserver started at http://127.1.132.129:38779/ using document root <none> and password file <none>
I20260923 22:16:24.439062 21322 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:24.439116 21322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:24.439246 21322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:24.440388 21322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data/instance:
uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "f17237acb7023206cf2a2f9542a6ae26"
server_key_iv: "c326803bdb3364540fa22dc8d6ca3110"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.440724 21322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance:
uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "f17237acb7023206cf2a2f9542a6ae26"
server_key_iv: "c326803bdb3364540fa22dc8d6ca3110"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.442013 21322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:24.442835 21336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.443058 21322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:24.443148 21322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal
uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "f17237acb7023206cf2a2f9542a6ae26"
server_key_iv: "c326803bdb3364540fa22dc8d6ca3110"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.443214 21322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:24.459631 21322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:24.459975 21322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:24.460114 21322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:24.460386 21322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:24.460800 21322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:24.460837 21322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.460866 21322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:24.460882 21322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.467320 21322 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.129:40101
I20260923 22:16:24.467360 21449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.129:40101 every 8 connection(s)
I20260923 22:16:24.467705 21322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/data/info.pb
I20260923 22:16:24.470816 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21322
I20260923 22:16:24.470903 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-0/wal/instance
I20260923 22:16:24.471086 1554 external_mini_cluster.cc:1428] Setting key db581d869d28182ce50005bf688c840c
I20260923 22:16:24.472165 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.130:0
--local_ip_for_outbound_sockets=127.1.132.130
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--builtin_ntp_servers=127.1.132.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:16:24.473474 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.468503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.129:45863 (local address 127.1.132.190:42971)
0923 22:16:24.468566 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.468569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.468786 (+ 217us) server_negotiation.cc:407] Connection header received
0923 22:16:24.469347 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.469349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.469366 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.469419 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.470132 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.470644 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.471813 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.472095 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.472157 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.473193 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.473194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.473195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:24.473205 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.473223 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:24.473228 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.473274 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.473356 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.473392 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:16:24.473742 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.468495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42971 (local address 127.1.132.129:45863)
0923 22:16:24.468663 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.468700 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.469232 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.469495 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.469499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.469507 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:24.470033 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.470038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.470741 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.470742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.471737 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.471741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.472209 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.472210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.472387 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.473023 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:24.473023 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:24.473058 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:24.473086 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:24.473282 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:24.473289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:24.473292 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.473359 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.473480 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260923 22:16:24.474236 21450 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42971
I20260923 22:16:24.474334 21450 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:24.474524 21450 heartbeater.cc:511] Master 127.1.132.190:42971 requested a full tablet report, sending...
I20260923 22:16:24.475031 21262 ts_manager.cc:194] Registered new tserver with Master: 8427e3d40d5a4cd5b66b2e88aa72c3e2 (127.1.132.129:40101)
I20260923 22:16:24.475658 21262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.129:45863
W20260923 22:16:24.560089 21453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:24.560367 21453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:24.560402 21453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:24.561725 21453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:24.561771 21453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:24.561808 21453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.130
I20260923 22:16:24.563236 21453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.132.130
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.130
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:24.563488 21453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:24.563791 21453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:16:24.568862 21453 server_base.cc:1034] running on GCE node
W20260923 22:16:24.568813 21461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.568804 21459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.568804 21458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.569226 21453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:24.569463 21453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:24.570626 21453 hybrid_clock.cc:648] HybridClock initialized: now 1790201784570609 us; error 31 us; skew 500 ppm
I20260923 22:16:24.571861 21453 webserver.cc:460] Webserver started at http://127.1.132.130:35567/ using document root <none> and password file <none>
I20260923 22:16:24.572114 21453 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:24.572199 21453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:24.572345 21453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:24.573393 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data/instance:
uuid: "309e9d5585b54f91bd607ca22e5e4121"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "0e91f97f01dbac8478840d3212e95dcc"
server_key_iv: "2119a7bc1950c598c56622fb1f7550c2"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.573734 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance:
uuid: "309e9d5585b54f91bd607ca22e5e4121"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "0e91f97f01dbac8478840d3212e95dcc"
server_key_iv: "2119a7bc1950c598c56622fb1f7550c2"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.575009 21453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:24.575779 21467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.575965 21453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:24.576047 21453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal
uuid: "309e9d5585b54f91bd607ca22e5e4121"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "0e91f97f01dbac8478840d3212e95dcc"
server_key_iv: "2119a7bc1950c598c56622fb1f7550c2"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.576118 21453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:24.583590 21453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:24.583900 21453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:24.584031 21453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:24.584295 21453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:24.584623 21453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:24.584658 21453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.584686 21453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:24.584709 21453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.590328 21453 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.130:38753
I20260923 22:16:24.590377 21580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.130:38753 every 8 connection(s)
I20260923 22:16:24.590711 21453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/data/info.pb
I20260923 22:16:24.596295 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.591419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:60941 (local address 127.1.132.190:42971)
0923 22:16:24.591492 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.591494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.591697 (+ 203us) server_negotiation.cc:407] Connection header received
0923 22:16:24.592230 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.592231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.592246 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.592314 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.593027 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.593611 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.594693 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.594890 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.594947 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.595890 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.595892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.595892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:24.595897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.595913 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:24.595917 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.595972 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.596156 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.596218 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:16:24.596531 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.591383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42971 (local address 127.1.132.130:60941)
0923 22:16:24.591557 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.591608 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.592115 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.592368 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.592371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.592380 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:24.592913 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.592919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.593720 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.593721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.594606 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.594610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.594981 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.594982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.595107 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.595729 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:24.595730 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:24.595761 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:24.595785 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:24.596039 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:24.596045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:24.596049 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.596127 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.596309 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260923 22:16:24.597066 21581 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42971
I20260923 22:16:24.597155 21581 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:24.597327 21581 heartbeater.cc:511] Master 127.1.132.190:42971 requested a full tablet report, sending...
I20260923 22:16:24.597514 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21453
I20260923 22:16:24.597612 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-1/wal/instance
I20260923 22:16:24.597690 21262 ts_manager.cc:194] Registered new tserver with Master: 309e9d5585b54f91bd607ca22e5e4121 (127.1.132.130:38753)
I20260923 22:16:24.597792 1554 external_mini_cluster.cc:1428] Setting key 24bbd3552bf186ae52ae271838c377e6
I20260923 22:16:24.598066 21262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.130:60941
I20260923 22:16:24.598786 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.132.131:0
--local_ip_for_outbound_sockets=127.1.132.131
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--builtin_ntp_servers=127.1.132.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:16:24.680472 21584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:24.680696 21584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:24.680727 21584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:24.681905 21584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:24.681949 21584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:24.681983 21584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.132.131
I20260923 22:16:24.683336 21584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.132.131
--webserver_port=0
--tserver_master_addrs=127.1.132.190:42971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.132.131
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:24.683583 21584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:24.683880 21584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:24.688899 21592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.688899 21590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:24.688994 21589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:24.688926 21584 server_base.cc:1034] running on GCE node
I20260923 22:16:24.689304 21584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:24.689512 21584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:24.690657 21584 hybrid_clock.cc:648] HybridClock initialized: now 1790201784690643 us; error 28 us; skew 500 ppm
I20260923 22:16:24.691956 21584 webserver.cc:460] Webserver started at http://127.1.132.131:39289/ using document root <none> and password file <none>
I20260923 22:16:24.692281 21584 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:24.692338 21584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:24.692461 21584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:24.693579 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data/instance:
uuid: "5a1ac570d7b440f39a7583d84d54f873"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "ecdbd62cb99b76db14bdb9c7f54f6e24"
server_key_iv: "b6567885d1fddfe9b96ac1e494d008d1"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.693934 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance:
uuid: "5a1ac570d7b440f39a7583d84d54f873"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "ecdbd62cb99b76db14bdb9c7f54f6e24"
server_key_iv: "b6567885d1fddfe9b96ac1e494d008d1"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.695209 21584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:24.695988 21598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.696255 21584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:24.696352 21584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal
uuid: "5a1ac570d7b440f39a7583d84d54f873"
format_stamp: "Formatted at 2026-09-23 22:16:24 on dist-test-slave-l4q5"
server_key: "ecdbd62cb99b76db14bdb9c7f54f6e24"
server_key_iv: "b6567885d1fddfe9b96ac1e494d008d1"
server_key_version: "encryptionkey@0"
I20260923 22:16:24.696422 21584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:24.719542 21584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:24.719861 21584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:24.719992 21584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:24.720237 21584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:16:24.720566 21584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:16:24.720598 21584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.720628 21584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:16:24.720644 21584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:24.725942 21584 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.131:44885
I20260923 22:16:24.725997 21711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.131:44885 every 8 connection(s)
I20260923 22:16:24.726296 21584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/data/info.pb
I20260923 22:16:24.731012 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.726985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.131:36705 (local address 127.1.132.190:42971)
0923 22:16:24.727048 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.727050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.727233 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:16:24.727691 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.727692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.727702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.727761 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.728402 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.728890 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.729827 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.730010 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.730058 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.730828 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.730829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.730829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:24.730833 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.730843 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:24.730847 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.730874 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.730932 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.730952 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:16:24.731196 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.726949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:42971 (local address 127.1.132.131:36705)
0923 22:16:24.727117 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.727159 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.727617 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.727819 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.727823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.727831 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:24.728334 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.728338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.728964 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.728965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.729736 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.729737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.730068 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.730068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.730175 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.730679 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:16:24.730679 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:24.730707 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:16:24.730731 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:24.730881 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:24.730883 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:24.730885 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.730928 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.731007 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260923 22:16:24.731635 21712 heartbeater.cc:347] Connected to a master server at 127.1.132.190:42971
I20260923 22:16:24.731726 21712 heartbeater.cc:464] Registering TS with master...
I20260923 22:16:24.731900 21712 heartbeater.cc:511] Master 127.1.132.190:42971 requested a full tablet report, sending...
I20260923 22:16:24.732270 21261 ts_manager.cc:194] Registered new tserver with Master: 5a1ac570d7b440f39a7583d84d54f873 (127.1.132.131:44885)
I20260923 22:16:24.732597 21261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.132.131:36705
I20260923 22:16:24.733989 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21584
I20260923 22:16:24.734071 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal/instance
I20260923 22:16:24.734217 1554 external_mini_cluster.cc:1428] Setting key c6f1fc0693b15cf13e9793eddf65440e
I20260923 22:16:24.735026 1554 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:16:24.738790 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.736310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.1.132.190:42971)
0923 22:16:24.736370 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.736372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.736474 (+ 102us) server_negotiation.cc:407] Connection header received
0923 22:16:24.736520 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.736520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.736532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.736576 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:24.736699 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.737170 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.737851 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.738025 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.738068 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.738619 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:24.738620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:24.738621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:24.738624 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:24.738637 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:24.738641 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:24.738677 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.738708 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.738730 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:16:24.740134 21261 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:16:24.741707 21261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:16:24.748266 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.1.132.131:44885)
0923 22:16:24.744690 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.744694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.744703 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:24.744737 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.744739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.744753 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.744819 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.744930 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.746180 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747264 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.747839 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747903 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748006 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.748112 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.748153 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":87,"threads_started":1}
I20260923 22:16:24.748288 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:38753 (local address 127.0.0.1:44354)
0923 22:16:24.744295 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.744350 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.744425 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.744610 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.744612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.744614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.744689 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.744691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.745852 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.745853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.747079 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.747083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.747955 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.748021 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748026 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.748068 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.748131 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260923 22:16:24.748266 21724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:44885 (local address 127.0.0.1:60880)
0923 22:16:24.744458 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.744467 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.744505 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.744818 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.744819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.744821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.744851 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.744852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.746283 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.746284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.747194 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.747197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.747919 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.748005 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748020 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.748093 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.748138 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":23,"thread_start_us":105,"threads_started":1}
I20260923 22:16:24.748709 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.1.132.130:38753)
0923 22:16:24.744434 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.744439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.744452 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:16:24.744513 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.744515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.744531 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.744609 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.744748 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.745781 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747163 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.747854 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.748466 (+ 612us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748560 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.748585 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.748627 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":85,"threads_started":1}
I20260923 22:16:24.748864 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:40101 (local address 127.0.0.1:55086)
0923 22:16:24.744533 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.744541 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.744574 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.744893 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.744894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.744895 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.744926 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.744927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.746271 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.746272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.747365 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.747368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.748625 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.748626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.748687 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748692 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.748744 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.748799 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260923 22:16:24.749127 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.744548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.1.132.129:40101)
0923 22:16:24.744725 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.744729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.744738 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:24.744785 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.744787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.744801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.744874 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.745004 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.746180 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.747445 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.748167 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.748261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.748350 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.749005 (+ 655us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.749050 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260923 22:16:24.749187 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 5f718e0871ac40a692ac1625bea4f094 (DEFAULT_TABLE table=test-table [id=b337d5ff4ede4cddae5b3e479255661e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:24.749339 21515 tablet_service.cc:1467] Processing CreateTablet for tablet 5f718e0871ac40a692ac1625bea4f094 (DEFAULT_TABLE table=test-table [id=b337d5ff4ede4cddae5b3e479255661e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:24.749557 21646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f718e0871ac40a692ac1625bea4f094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:24.749603 21515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f718e0871ac40a692ac1625bea4f094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:24.752982 21727 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Bootstrap starting.
I20260923 22:16:24.753053 21384 tablet_service.cc:1467] Processing CreateTablet for tablet 5f718e0871ac40a692ac1625bea4f094 (DEFAULT_TABLE table=test-table [id=b337d5ff4ede4cddae5b3e479255661e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:16:24.753266 21726 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Bootstrap starting.
I20260923 22:16:24.753423 21384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f718e0871ac40a692ac1625bea4f094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:24.754006 21726 tablet_bootstrap.cc:654] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:24.754359 21726 log.cc:824] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:24.755084 21726 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: No bootstrap required, opened a new log
I20260923 22:16:24.755203 21726 ts_tablet_manager.cc:1397] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:24.755311 21727 tablet_bootstrap.cc:654] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:24.755640 21727 log.cc:824] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:24.756500 21727 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: No bootstrap required, opened a new log
I20260923 22:16:24.756577 21727 ts_tablet_manager.cc:1397] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:16:24.757169 21730 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: Bootstrap starting.
I20260923 22:16:24.757683 21730 tablet_bootstrap.cc:654] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:24.757936 21730 log.cc:824] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:24.757830 21726 raft_consensus.cc:348] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.757997 21726 raft_consensus.cc:374] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:24.758039 21726 raft_consensus.cc:729] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309e9d5585b54f91bd607ca22e5e4121, State: Initialized, Role: FOLLOWER
I20260923 22:16:24.758177 21726 consensus_queue.cc:260] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.758447 21726 ts_tablet_manager.cc:1428] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 22:16:24.758559 21581 heartbeater.cc:503] Master 127.1.132.190:42971 was elected leader, sending a full tablet report...
I20260923 22:16:24.758764 21730 tablet_bootstrap.cc:492] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: No bootstrap required, opened a new log
I20260923 22:16:24.758828 21730 ts_tablet_manager.cc:1397] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:16:24.759071 21727 raft_consensus.cc:348] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.759248 21727 raft_consensus.cc:374] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:24.759289 21727 raft_consensus.cc:729] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ac570d7b440f39a7583d84d54f873, State: Initialized, Role: FOLLOWER
I20260923 22:16:24.759503 21727 consensus_queue.cc:260] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.759740 21727 ts_tablet_manager.cc:1428] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:16:24.759794 21712 heartbeater.cc:503] Master 127.1.132.190:42971 was elected leader, sending a full tablet report...
I20260923 22:16:24.761174 21730 raft_consensus.cc:348] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.761317 21730 raft_consensus.cc:374] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:24.761346 21730 raft_consensus.cc:729] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8427e3d40d5a4cd5b66b2e88aa72c3e2, State: Initialized, Role: FOLLOWER
I20260923 22:16:24.761469 21730 consensus_queue.cc:260] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.761719 21730 ts_tablet_manager.cc:1428] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:16:24.761837 21450 heartbeater.cc:503] Master 127.1.132.190:42971 was elected leader, sending a full tablet report...
W20260923 22:16:24.841253 21582 tablet.cc:2369] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:16:24.886996 21732 raft_consensus.cc:482] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:16:24.887122 21732 raft_consensus.cc:504] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.887475 21732 leader_election.cc:288] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a1ac570d7b440f39a7583d84d54f873 (127.1.132.131:44885), 8427e3d40d5a4cd5b66b2e88aa72c3e2 (127.1.132.129:40101)
I20260923 22:16:24.890448 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.887735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:40917 (local address 127.1.132.131:44885)
0923 22:16:24.887784 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.887786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.887796 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:24.887838 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.887839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.887847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.887924 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.888168 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.888730 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.889778 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.890134 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.890195 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.890203 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.890342 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.890380 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:16:24.890479 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.887646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.131:44885 (local address 127.1.132.130:40917)
0923 22:16:24.887699 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.887711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.887759 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.887929 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.887930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.887932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.887990 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.887992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.888809 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.888810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.889697 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.889700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.890215 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.890216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.890284 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.890287 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.890331 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.890374 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:16:24.890504 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.887755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:40101 (local address 127.1.132.130:50465)
0923 22:16:24.887929 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.887940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:24.887997 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:24.888156 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:24.888157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:24.888159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:16:24.888249 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.888251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.888878 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.888879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.889714 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:24.889716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.890234 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.890235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:24.890316 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.890319 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:16:24.890366 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:16:24.890403 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260923 22:16:24.890486 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.887821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.132.130:50465 (local address 127.1.132.129:40101)
0923 22:16:24.887870 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.887872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.887986 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:16:24.888057 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.888058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.888071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.888117 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:16:24.888315 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.888805 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.889805 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.890161 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.890219 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.890227 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.890381 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.890421 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:16:24.890836 21404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f718e0871ac40a692ac1625bea4f094" candidate_uuid: "309e9d5585b54f91bd607ca22e5e4121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" is_pre_election: true
I20260923 22:16:24.890851 21666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f718e0871ac40a692ac1625bea4f094" candidate_uuid: "309e9d5585b54f91bd607ca22e5e4121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ac570d7b440f39a7583d84d54f873" is_pre_election: true
I20260923 22:16:24.890981 21404 raft_consensus.cc:2454] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 309e9d5585b54f91bd607ca22e5e4121 in term 0.
I20260923 22:16:24.890988 21666 raft_consensus.cc:2454] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 309e9d5585b54f91bd607ca22e5e4121 in term 0.
I20260923 22:16:24.891201 21470 leader_election.cc:302] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309e9d5585b54f91bd607ca22e5e4121, 8427e3d40d5a4cd5b66b2e88aa72c3e2; no voters:
I20260923 22:16:24.891362 21732 raft_consensus.cc:2781] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:16:24.891439 21732 raft_consensus.cc:482] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:16:24.891458 21732 raft_consensus.cc:3037] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.892167 21732 raft_consensus.cc:504] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.892335 21732 leader_election.cc:288] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [CANDIDATE]: Term 1 election: Requested vote from peers 5a1ac570d7b440f39a7583d84d54f873 (127.1.132.131:44885), 8427e3d40d5a4cd5b66b2e88aa72c3e2 (127.1.132.129:40101)
I20260923 22:16:24.892481 21666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f718e0871ac40a692ac1625bea4f094" candidate_uuid: "309e9d5585b54f91bd607ca22e5e4121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ac570d7b440f39a7583d84d54f873"
I20260923 22:16:24.892557 21404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f718e0871ac40a692ac1625bea4f094" candidate_uuid: "309e9d5585b54f91bd607ca22e5e4121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2"
I20260923 22:16:24.892562 21666 raft_consensus.cc:3037] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.892642 21404 raft_consensus.cc:3037] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:24.893343 21666 raft_consensus.cc:2454] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 309e9d5585b54f91bd607ca22e5e4121 in term 1.
I20260923 22:16:24.893388 21404 raft_consensus.cc:2454] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 309e9d5585b54f91bd607ca22e5e4121 in term 1.
I20260923 22:16:24.893504 21470 leader_election.cc:302] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309e9d5585b54f91bd607ca22e5e4121, 8427e3d40d5a4cd5b66b2e88aa72c3e2; no voters:
I20260923 22:16:24.893600 21732 raft_consensus.cc:2781] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:24.893782 21732 raft_consensus.cc:686] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 1 LEADER]: Becoming Leader. State: Replica: 309e9d5585b54f91bd607ca22e5e4121, State: Running, Role: LEADER
I20260923 22:16:24.893882 21732 consensus_queue.cc:237] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } }
I20260923 22:16:24.894644 21261 catalog_manager.cc:5495] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 reported cstate change: term changed from 0 to 1, leader changed from <none> to 309e9d5585b54f91bd607ca22e5e4121 (127.1.132.130). New cstate: current_term: 1 leader_uuid: "309e9d5585b54f91bd607ca22e5e4121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "309e9d5585b54f91bd607ca22e5e4121" member_type: VOTER last_known_addr { host: "127.1.132.130" port: 38753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 } health_report { overall_health: UNKNOWN } } }
I20260923 22:16:24.929509 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.927221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.1.132.130:38753)
0923 22:16:24.927285 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:24.927287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:24.927297 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:24.927342 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:24.927343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:24.927361 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:24.927421 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:24.927539 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.928006 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.928801 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:24.928985 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:24.929040 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:24.929180 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:24.929290 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:24.929364 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:24.929384 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 22:16:24.929404 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:16:24.931432 21404 raft_consensus.cc:1261] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 1 FOLLOWER]: Refusing update from remote peer 309e9d5585b54f91bd607ca22e5e4121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:24.931458 21666 raft_consensus.cc:1261] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 1 FOLLOWER]: Refusing update from remote peer 309e9d5585b54f91bd607ca22e5e4121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:24.931818 21736 consensus_queue.cc:1035] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8427e3d40d5a4cd5b66b2e88aa72c3e2" member_type: VOTER last_known_addr { host: "127.1.132.129" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:24.931962 21732 consensus_queue.cc:1035] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1ac570d7b440f39a7583d84d54f873" member_type: VOTER last_known_addr { host: "127.1.132.131" port: 44885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:24.936900 21262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:24.937002 21262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:16:24.937690 21262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 045dc4d0b9f2423baa833fb1c02742f6: Sending DeleteTablet for 3 replicas of tablet 5f718e0871ac40a692ac1625bea4f094
I20260923 22:16:24.937994 21515 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f718e0871ac40a692ac1625bea4f094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='slave'} at 127.0.0.1:44354
I20260923 22:16:24.937989 21646 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f718e0871ac40a692ac1625bea4f094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='slave'} at 127.0.0.1:60880
I20260923 22:16:24.938062 21384 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f718e0871ac40a692ac1625bea4f094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:16:24 UTC) from {username='slave'} at 127.0.0.1:55086
I20260923 22:16:24.938253 21749 tablet_replica.cc:330] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: stopping tablet replica
I20260923 22:16:24.938375 21749 raft_consensus.cc:2229] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:24.938446 21751 tablet_replica.cc:330] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2: stopping tablet replica
I20260923 22:16:24.938514 21749 raft_consensus.cc:2258] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:16:24.938532 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21322
I20260923 22:16:24.938555 21751 raft_consensus.cc:2229] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:16:24.938644 21751 raft_consensus.cc:2258] T 5f718e0871ac40a692ac1625bea4f094 P 8427e3d40d5a4cd5b66b2e88aa72c3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:16:24.943764 21248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.132.129:40101 (error 108)
I20260923 22:16:24.944473 21750 tablet_replica.cc:330] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121: stopping tablet replica
I20260923 22:16:24.944653 21750 raft_consensus.cc:2229] T 5f718e0871ac40a692ac1625bea4f094 P 309e9d5585b54f91bd607ca22e5e4121 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:16:24.944721 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21453
I20260923 22:16:24.944799 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.944640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.129:40101 (local address 127.0.0.1:55100)
0923 22:16:24.944694 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.944762 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.129:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:16:24.945266 21248 catalog_manager.cc:4584] TS 8427e3d40d5a4cd5b66b2e88aa72c3e2 (127.1.132.129:40101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f718e0871ac40a692ac1625bea4f094: Network error: Client connection negotiation failed: client connection to 127.1.132.129:40101: connect: Connection refused (error 111)
I20260923 22:16:24.946264 21749 ts_tablet_manager.cc:1905] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:16:24.947731 21749 ts_tablet_manager.cc:1918] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:16:24.947795 21749 log.cc:1196] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Deleting WAL directory at /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201541420007-1554-0/minicluster-data/ts-2/wal/wals/5f718e0871ac40a692ac1625bea4f094
I20260923 22:16:24.948004 21749 ts_tablet_manager.cc:1939] T 5f718e0871ac40a692ac1625bea4f094 P 5a1ac570d7b440f39a7583d84d54f873: Deleting consensus metadata
I20260923 22:16:24.948467 21248 catalog_manager.cc:4841] TS 5a1ac570d7b440f39a7583d84d54f873 (127.1.132.131:44885): tablet 5f718e0871ac40a692ac1625bea4f094 (table test-table [id=b337d5ff4ede4cddae5b3e479255661e]) successfully deleted
I20260923 22:16:24.949376 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21584
I20260923 22:16:24.949397 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:24.949297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.130:38753 (local address 127.0.0.1:44368)
0923 22:16:24.949341 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:24.949369 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.130:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:16:24.953300 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21231
2026-09-23T22:16:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (812 ms)
[----------] 6 tests from EncryptionPolicyTest (8477 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:25 dist-test-slave-l4q5 krb5kdc[21755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:25 dist-test-slave-l4q5 krb5kdc[21755](info): set up 2 sockets
Sep 23 22:16:25 dist-test-slave-l4q5 krb5kdc[21755](info): commencing operation
krb5kdc: starting...
W20260923 22:16:25.915874 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:25 dist-test-slave-l4q5 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:26.034991 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:26.115594 21768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:26.115844 21768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:26.115877 21768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:26.117187 21768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:26.117238 21768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:26.117257 21768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:26.117272 21768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:26.117290 21768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:26.118865 21768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:26.119118 21768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:26.119331 21768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:26.123888 21773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:26.124013 21776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:26.124013 21768 server_base.cc:1034] running on GCE node
W20260923 22:16:26.123894 21774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:26.124402 21768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:26.124693 21768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:26.125835 21768 hybrid_clock.cc:648] HybridClock initialized: now 1790201786125789 us; error 63 us; skew 500 ppm
Sep 23 22:16:26 dist-test-slave-l4q5 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201786, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:26.136736 21768 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:26.137179 21768 webserver.cc:460] Webserver started at http://127.0.0.1:45125/ using document root <none> and password file <none>
I20260923 22:16:26.137418 21768 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:26.137465 21768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:26.137586 21768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:26.138643 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "285dde2a574e4315a1411edb53c98250"
format_stamp: "Formatted at 2026-09-23 22:16:26 on dist-test-slave-l4q5"
server_key: "ba93806b3d086ff16e67a76bb335f3da"
server_key_iv: "75847e3f27809cf1b8f8f16f0561bf99"
server_key_version: "encryptionkey@0"
I20260923 22:16:26.138970 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "285dde2a574e4315a1411edb53c98250"
format_stamp: "Formatted at 2026-09-23 22:16:26 on dist-test-slave-l4q5"
server_key: "ba93806b3d086ff16e67a76bb335f3da"
server_key_iv: "75847e3f27809cf1b8f8f16f0561bf99"
server_key_version: "encryptionkey@0"
I20260923 22:16:26.140336 21768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:26.141078 21783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:26.141266 21768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:26.141340 21768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "285dde2a574e4315a1411edb53c98250"
format_stamp: "Formatted at 2026-09-23 22:16:26 on dist-test-slave-l4q5"
server_key: "ba93806b3d086ff16e67a76bb335f3da"
server_key_iv: "75847e3f27809cf1b8f8f16f0561bf99"
server_key_version: "encryptionkey@0"
I20260923 22:16:26.141417 21768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:26.173611 21768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:26.175490 21768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:26.175621 21768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:26.179477 21768 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37291
I20260923 22:16:26.179518 21835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37291 every 8 connection(s)
I20260923 22:16:26.179889 21768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:26.180887 21836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:26.183189 21836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250: Bootstrap starting.
I20260923 22:16:26.183758 21836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:26.184023 21836 log.cc:824] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:26.184795 21836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250: No bootstrap required, opened a new log
I20260923 22:16:26.186403 21836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } }
I20260923 22:16:26.186553 21836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:26.186573 21836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285dde2a574e4315a1411edb53c98250, State: Initialized, Role: FOLLOWER
I20260923 22:16:26.186703 21836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } }
I20260923 22:16:26.186774 21836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:26.186795 21836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:26.186838 21836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:26.187386 21836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } }
I20260923 22:16:26.187474 21836 leader_election.cc:302] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 285dde2a574e4315a1411edb53c98250; no voters:
I20260923 22:16:26.187629 21836 leader_election.cc:288] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:26.187674 21839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:26.187884 21836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:26.187896 21839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [term 1 LEADER]: Becoming Leader. State: Replica: 285dde2a574e4315a1411edb53c98250, State: Running, Role: LEADER
I20260923 22:16:26.188023 21839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } }
I20260923 22:16:26.188417 21841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 285dde2a574e4315a1411edb53c98250. Latest consensus state: current_term: 1 leader_uuid: "285dde2a574e4315a1411edb53c98250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } } }
I20260923 22:16:26.188499 21841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:26.188548 21840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "285dde2a574e4315a1411edb53c98250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285dde2a574e4315a1411edb53c98250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37291 } } }
I20260923 22:16:26.188616 21840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:26.188712 21848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:26.189219 21848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:26.190101 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21768
I20260923 22:16:26.190203 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:26.190372 1554 external_mini_cluster.cc:1428] Setting key 90b9aa41172245db444d8d41991fd9f0
I20260923 22:16:26.190774 21848 catalog_manager.cc:1347] Generated new cluster ID: 9dcfe14fc3ba4a18918087d8b4d675c4
I20260923 22:16:26.190824 21848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:26 dist-test-slave-l4q5 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:26.202852 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:26.191044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40519 (local address 127.0.0.1:37291)
0923 22:16:26.191198 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:26.191208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:26.191241 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:16:26.191362 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:26.191369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:26.191424 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:26.191566 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:26.192249 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:26.193693 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:26.194765 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:26.195426 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:26.195545 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:26.198749 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:26.198755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:26.198755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:26.198770 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:26.200828 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:26.201380 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:26.201384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:26.201384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:26.201504 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:26.201824 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:26.201825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:26.201825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:26.202147 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:26.202301 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:26.202496 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 22:16:26.202602 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260923 22:16:26.204169 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:26.205272 21848 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:26.205931 21848 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:26.212646 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:26.205593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46429 (local address 127.0.0.1:37291)
0923 22:16:26.205627 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:26.205628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:26.205637 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:26.205680 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:26.205681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:26.205691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:26.205744 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:26.206421 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:26.207239 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:26.208284 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:26.208614 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:26.208678 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:26.210009 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:26.210011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:26.210011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:26.210018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:26.211520 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:26.211899 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:26.211900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:26.211901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:26.211958 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:26.212139 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:26.212140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:26.212140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:26.212285 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:26.212385 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:26.212487 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:16:26.212549 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:16:26.217829 21848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 285dde2a574e4315a1411edb53c98250: Generated new TSK 0
I20260923 22:16:26.218034 21848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:26.220284 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21768
W20260923 22:16:26.223963 21753 connection.cc:535] client connection to 127.0.0.1:37291 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37291): Connection reset by peer (error 104)
2026-09-23T22:16:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:26 dist-test-slave-l4q5 krb5kdc[21867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:26 dist-test-slave-l4q5 krb5kdc[21867](info): set up 2 sockets
Sep 23 22:16:26 dist-test-slave-l4q5 krb5kdc[21867](info): commencing operation
krb5kdc: starting...
W20260923 22:16:27.171983 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:27 dist-test-slave-l4q5 krb5kdc[21867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:27.290884 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:27.370054 21880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:27.370280 21880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:27.370303 21880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:27.371485 21880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:27.371529 21880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:27.371542 21880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:27.371553 21880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:27.371564 21880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:27.373051 21880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:27.373281 21880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:27.373478 21880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:27.378162 21886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:27.378182 21888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:27.378245 21880 server_base.cc:1034] running on GCE node
W20260923 22:16:27.378170 21885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:27.378609 21880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:27.378901 21880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:27.380075 21880 hybrid_clock.cc:648] HybridClock initialized: now 1790201787380032 us; error 60 us; skew 500 ppm
Sep 23 22:16:27 dist-test-slave-l4q5 krb5kdc[21867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:27.382651 21880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:27.383118 21880 webserver.cc:460] Webserver started at http://127.0.0.1:37831/ using document root <none> and password file <none>
I20260923 22:16:27.383324 21880 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:27.383374 21880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:27.383502 21880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:27.384671 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "40521fe137204be8b4342bf904a2047d"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-l4q5"
server_key: "8dd6d4210f2d67d07b9c0c835ed7a0b3"
server_key_iv: "1a2986c9fdfbaf47713a305a5daf5e1a"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.385031 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "40521fe137204be8b4342bf904a2047d"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-l4q5"
server_key: "8dd6d4210f2d67d07b9c0c835ed7a0b3"
server_key_iv: "1a2986c9fdfbaf47713a305a5daf5e1a"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.386354 21880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:27.387076 21895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:27.387239 21880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:27.387332 21880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "40521fe137204be8b4342bf904a2047d"
format_stamp: "Formatted at 2026-09-23 22:16:27 on dist-test-slave-l4q5"
server_key: "8dd6d4210f2d67d07b9c0c835ed7a0b3"
server_key_iv: "1a2986c9fdfbaf47713a305a5daf5e1a"
server_key_version: "encryptionkey@0"
I20260923 22:16:27.387398 21880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:27.411903 21880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:27.413771 21880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:27.413913 21880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:27.417629 21880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33255
I20260923 22:16:27.417662 21947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33255 every 8 connection(s)
I20260923 22:16:27.418004 21880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:27.418902 21948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:27.421134 21948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d: Bootstrap starting.
I20260923 22:16:27.421671 21948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:27.421913 21948 log.cc:824] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:27.422531 21948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d: No bootstrap required, opened a new log
I20260923 22:16:27.424000 21948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } }
I20260923 22:16:27.424129 21948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:27.424150 21948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40521fe137204be8b4342bf904a2047d, State: Initialized, Role: FOLLOWER
I20260923 22:16:27.424319 21948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } }
I20260923 22:16:27.424381 21948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:27.424400 21948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:27.424443 21948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:27.424942 21948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } }
I20260923 22:16:27.425024 21948 leader_election.cc:302] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40521fe137204be8b4342bf904a2047d; no voters:
I20260923 22:16:27.425172 21948 leader_election.cc:288] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:27.425220 21951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:27.425369 21951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [term 1 LEADER]: Becoming Leader. State: Replica: 40521fe137204be8b4342bf904a2047d, State: Running, Role: LEADER
I20260923 22:16:27.425416 21948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:27.425460 21951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } }
I20260923 22:16:27.425693 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21880
I20260923 22:16:27.425796 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:27.425894 21953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40521fe137204be8b4342bf904a2047d. Latest consensus state: current_term: 1 leader_uuid: "40521fe137204be8b4342bf904a2047d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } } }
I20260923 22:16:27.425961 21953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:27.425972 1554 external_mini_cluster.cc:1428] Setting key a7fcfe0b25074dfa51b626a974fd8a99
I20260923 22:16:27.426306 21952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40521fe137204be8b4342bf904a2047d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40521fe137204be8b4342bf904a2047d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33255 } } }
I20260923 22:16:27.426368 21952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:27.426607 21960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:27.427193 21960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:27.429028 21960 catalog_manager.cc:1347] Generated new cluster ID: 870e7b6ab0b543c894b1c0d83c46c257
I20260923 22:16:27.429078 21960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:27.431259 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:27.426482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59627 (local address 127.0.0.1:33255)
0923 22:16:27.426659 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:27.426668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:27.426700 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:16:27.426815 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:27.426822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:27.426877 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:27.427025 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:27.427244 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:27.428614 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:27.429461 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:27.429892 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:27.429989 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:27.430634 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:27.430638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:27.430638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:27.430802 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:27.430833 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:27.430844 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:27.430920 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:27.431027 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 22:16:27.431057 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260923 22:16:27.433084 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:27.439239 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:27.436518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35357 (local address 127.0.0.1:33255)
0923 22:16:27.436557 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:27.436559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:27.436676 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:16:27.436743 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:27.436743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:27.436754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:27.436797 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:27.436924 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:27.437439 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:27.438156 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:27.438369 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:27.438424 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:27.439014 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:27.439015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:27.439015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:27.439019 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:27.439034 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:27.439038 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:27.439085 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:27.439149 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 22:16:27.439161 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:16:27.450237 21960 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:27.450805 21960 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:27.453986 21960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40521fe137204be8b4342bf904a2047d: Generated new TSK 0
I20260923 22:16:27.454156 21960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:27.458596 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21880
W20260923 22:16:27.462466 21865 connection.cc:535] client connection to 127.0.0.1:33255 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33255): Connection reset by peer (error 104)
2026-09-23T22:16:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:27 dist-test-slave-l4q5 krb5kdc[21979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:27 dist-test-slave-l4q5 krb5kdc[21979](info): set up 2 sockets
Sep 23 22:16:27 dist-test-slave-l4q5 krb5kdc[21979](info): commencing operation
krb5kdc: starting...
W20260923 22:16:28.419947 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:28 dist-test-slave-l4q5 krb5kdc[21979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:28.538780 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:28.618011 21992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:28.618218 21992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:28.618239 21992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:28.619395 21992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:28.619441 21992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:28.619452 21992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:28.619464 21992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:28.619475 21992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:28.621151 21992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:28.621382 21992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:28.621604 21992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:28.626260 21998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:28.626267 21997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:28.626396 22000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:28.626621 21992 server_base.cc:1034] running on GCE node
I20260923 22:16:28.626837 21992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:28.627125 21992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:28.628283 21992 hybrid_clock.cc:648] HybridClock initialized: now 1790201788628266 us; error 29 us; skew 500 ppm
Sep 23 22:16:28 dist-test-slave-l4q5 krb5kdc[21979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201788, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:28.630811 21992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:28.631261 21992 webserver.cc:460] Webserver started at http://127.0.0.1:42101/ using document root <none> and password file <none>
I20260923 22:16:28.631469 21992 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:28.631524 21992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:28.631649 21992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:28.632725 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "da4ba85077934a7bad96351a28c80f6e"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-l4q5"
server_key: "5361051726157eb7890bea9a5f878dec"
server_key_iv: "ad24b9ce6409fa9fba0e9024de20941b"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.633078 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "da4ba85077934a7bad96351a28c80f6e"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-l4q5"
server_key: "5361051726157eb7890bea9a5f878dec"
server_key_iv: "ad24b9ce6409fa9fba0e9024de20941b"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.634375 21992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:28.635159 22007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:28.635325 21992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:28.635408 21992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "da4ba85077934a7bad96351a28c80f6e"
format_stamp: "Formatted at 2026-09-23 22:16:28 on dist-test-slave-l4q5"
server_key: "5361051726157eb7890bea9a5f878dec"
server_key_iv: "ad24b9ce6409fa9fba0e9024de20941b"
server_key_version: "encryptionkey@0"
I20260923 22:16:28.635470 21992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:28.638772 21992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:28.640875 21992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:28.641006 21992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:28.644620 21992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35049
I20260923 22:16:28.644663 22059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35049 every 8 connection(s)
I20260923 22:16:28.644969 21992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:28.645941 22060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:28.648541 22060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: Bootstrap starting.
I20260923 22:16:28.649096 22060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:28.649338 22060 log.cc:824] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:28.650040 22060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: No bootstrap required, opened a new log
I20260923 22:16:28.651644 22060 raft_consensus.cc:348] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260923 22:16:28.651784 22060 raft_consensus.cc:374] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:28.651806 22060 raft_consensus.cc:729] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da4ba85077934a7bad96351a28c80f6e, State: Initialized, Role: FOLLOWER
I20260923 22:16:28.651929 22060 consensus_queue.cc:260] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260923 22:16:28.651993 22060 raft_consensus.cc:388] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:28.652024 22060 raft_consensus.cc:482] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:28.652071 22060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:28.652634 22060 raft_consensus.cc:504] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260923 22:16:28.652730 22060 leader_election.cc:302] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da4ba85077934a7bad96351a28c80f6e; no voters:
I20260923 22:16:28.652894 22060 leader_election.cc:288] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:28.652937 22063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:28.653110 22063 raft_consensus.cc:686] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [term 1 LEADER]: Becoming Leader. State: Replica: da4ba85077934a7bad96351a28c80f6e, State: Running, Role: LEADER
I20260923 22:16:28.653134 22060 sys_catalog.cc:564] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:28.653278 22063 consensus_queue.cc:237] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260923 22:16:28.653403 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 21992
I20260923 22:16:28.653573 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:28.653782 1554 external_mini_cluster.cc:1428] Setting key 794b2f3d0c3f549da321c0b075ada7c6
I20260923 22:16:28.653915 22065 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader da4ba85077934a7bad96351a28c80f6e. Latest consensus state: current_term: 1 leader_uuid: "da4ba85077934a7bad96351a28c80f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260923 22:16:28.653990 22065 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:28.654397 22064 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da4ba85077934a7bad96351a28c80f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da4ba85077934a7bad96351a28c80f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260923 22:16:28.654601 22064 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:28.655419 22080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:28.655471 22080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:28.655679 22078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:28.655855 22078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:28.656248 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:28.654465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41153 (local address 127.0.0.1:35049)
0923 22:16:28.654639 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:28.654650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:28.654687 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:16:28.654815 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:28.654821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:28.654856 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:28.655032 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:28.655712 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:28.655715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:28.655715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:28.655720 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:28.655764 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:28.655779 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:28.655843 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:28.655901 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:16:28.656045 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":95,"threads_started":1}
I20260923 22:16:28.657277 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:28.657399 22078 catalog_manager.cc:1347] Generated new cluster ID: 44588e395652496f8270350e932fb32a
I20260923 22:16:28.657437 22078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:28.659494 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:28.658527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42121 (local address 127.0.0.1:35049)
0923 22:16:28.658567 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:28.658568 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:28.658683 (+ 115us) server_negotiation.cc:407] Connection header received
0923 22:16:28.658736 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:28.658737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:28.658742 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:28.658793 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:28.659306 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:28.659307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:28.659307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:28.659311 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:28.659327 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:28.659331 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:28.659373 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:28.659420 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:16:28.659433 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:28.677673 22078 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:28.677881 22078 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:28.682240 22078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da4ba85077934a7bad96351a28c80f6e: Generated new TSK 0
I20260923 22:16:28.682430 22078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:28.684998 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 21992
2026-09-23T22:16:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:28 dist-test-slave-l4q5 krb5kdc[22091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:28 dist-test-slave-l4q5 krb5kdc[22091](info): set up 2 sockets
Sep 23 22:16:28 dist-test-slave-l4q5 krb5kdc[22091](info): commencing operation
krb5kdc: starting...
W20260923 22:16:29.644016 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:29.763772 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:29.843698 22104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:29.843926 22104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:29.843961 22104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:29.845217 22104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:29.845270 22104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:29.845289 22104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:29.845306 22104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:29.845321 22104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:29.846884 22104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:29.847147 22104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:29.847359 22104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:29.852109 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:29.852103 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:29.852097 22110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:29.852120 22104 server_base.cc:1034] running on GCE node
I20260923 22:16:29.852499 22104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:29.852778 22104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:29.853925 22104 hybrid_clock.cc:648] HybridClock initialized: now 1790201789853912 us; error 29 us; skew 500 ppm
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:29.856360 22104 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:29.856843 22104 webserver.cc:460] Webserver started at http://127.0.0.1:42749/ using document root <none> and password file <none>
I20260923 22:16:29.857066 22104 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:29.857115 22104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:29.857241 22104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:29.858259 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "1c4f20412dc74b298bcc1252ed4d6477"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-l4q5"
server_key: "5186b0679f0d5b33cb8977df726d610a"
server_key_iv: "cb2a6655f63e09bcf9d16bcf5524118c"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.858580 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "1c4f20412dc74b298bcc1252ed4d6477"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-l4q5"
server_key: "5186b0679f0d5b33cb8977df726d610a"
server_key_iv: "cb2a6655f63e09bcf9d16bcf5524118c"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.859814 22104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:29.860548 22119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:29.860699 22104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:29.860778 22104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "1c4f20412dc74b298bcc1252ed4d6477"
format_stamp: "Formatted at 2026-09-23 22:16:29 on dist-test-slave-l4q5"
server_key: "5186b0679f0d5b33cb8977df726d610a"
server_key_iv: "cb2a6655f63e09bcf9d16bcf5524118c"
server_key_version: "encryptionkey@0"
I20260923 22:16:29.860843 22104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:29.883495 22104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:29.885684 22104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:29.885819 22104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:29.889477 22104 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44585
I20260923 22:16:29.889539 22171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44585 every 8 connection(s)
I20260923 22:16:29.889842 22104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:29.890817 22172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:29.893088 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477: Bootstrap starting.
I20260923 22:16:29.893651 22172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:29.893896 22172 log.cc:824] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:29.894606 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477: No bootstrap required, opened a new log
I20260923 22:16:29.896142 22172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260923 22:16:29.896309 22172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:29.896332 22172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4f20412dc74b298bcc1252ed4d6477, State: Initialized, Role: FOLLOWER
I20260923 22:16:29.896492 22172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260923 22:16:29.896560 22172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:29.896580 22172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:29.896610 22172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:29.897117 22172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260923 22:16:29.897205 22172 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4f20412dc74b298bcc1252ed4d6477; no voters:
I20260923 22:16:29.897353 22172 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:29.897431 22175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:29.897593 22175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4f20412dc74b298bcc1252ed4d6477, State: Running, Role: LEADER
I20260923 22:16:29.897593 22172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:29.897740 22175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } }
I20260923 22:16:29.898136 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4f20412dc74b298bcc1252ed4d6477. Latest consensus state: current_term: 1 leader_uuid: "1c4f20412dc74b298bcc1252ed4d6477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260923 22:16:29.898288 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:29.898164 22176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4f20412dc74b298bcc1252ed4d6477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4f20412dc74b298bcc1252ed4d6477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44585 } } }
I20260923 22:16:29.898507 22176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:29.898537 22179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:29.898820 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22104
I20260923 22:16:29.898953 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:29.899085 22179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:29.899142 1554 external_mini_cluster.cc:1428] Setting key 7bac9a4db5277119e1a35df558474b20
I20260923 22:16:29.900826 22179 catalog_manager.cc:1347] Generated new cluster ID: 99b4923cddf742279d54eec3f8aded4d
I20260923 22:16:29.900880 22179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:29.912048 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:29.899865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35375 (local address 127.0.0.1:44585)
0923 22:16:29.900434 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:29.900445 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:29.900479 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:16:29.900621 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:29.900627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:29.900683 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:29.900814 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:29.901531 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.902994 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.903765 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.904434 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.904539 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:29.907680 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:29.907686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:29.907686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:29.907701 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:29.910264 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:29.910684 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:29.910688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:29.910689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:29.910838 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:29.911036 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:29.911037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:29.911038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:29.911399 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:29.911563 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:29.911637 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:16:29.911798 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":102,"threads_started":1}
I20260923 22:16:29.913043 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:29.915915 22179 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:29.916595 22179 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:29.921609 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:29.914515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58297 (local address 127.0.0.1:44585)
0923 22:16:29.914553 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:29.914555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:29.914663 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:16:29.914712 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:29.914713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:29.914724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:29.914784 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:29.915386 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.916206 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.917140 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:29.917468 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:29.917537 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:29.918844 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:29.918846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:29.918846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:29.918853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:29.920431 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:29.920802 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:29.920803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:29.920804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:29.920862 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:29.921046 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:29.921047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:29.921047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:29.921181 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:29.921273 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:29.921365 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:16:29.921496 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:29.924028 22179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4f20412dc74b298bcc1252ed4d6477: Generated new TSK 0
I20260923 22:16:29.924248 22179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:29.929286 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22104
2026-09-23T22:16:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22203](info): set up 2 sockets
Sep 23 22:16:29 dist-test-slave-l4q5 krb5kdc[22203](info): commencing operation
krb5kdc: starting...
W20260923 22:16:30.892050 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:30 dist-test-slave-l4q5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:31.012884 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:31.093351 22216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:31.093585 22216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:31.093607 22216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:31.094805 22216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:31.094851 22216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:31.094864 22216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:31.094877 22216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:31.094887 22216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:31.096467 22216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:31.096702 22216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:31.096927 22216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:31.101810 22221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:31.101804 22222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:31.101856 22216 server_base.cc:1034] running on GCE node
W20260923 22:16:31.101805 22224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:31.102244 22216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:31.102524 22216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:31.103678 22216 hybrid_clock.cc:648] HybridClock initialized: now 1790201791103657 us; error 32 us; skew 500 ppm
Sep 23 22:16:31 dist-test-slave-l4q5 krb5kdc[22203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:31.106308 22216 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:31.106730 22216 webserver.cc:460] Webserver started at http://127.0.0.1:41119/ using document root <none> and password file <none>
I20260923 22:16:31.106938 22216 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:31.106987 22216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:31.107108 22216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:31.108173 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "6276b66869d94ef39f4a6e00209a812f"
format_stamp: "Formatted at 2026-09-23 22:16:31 on dist-test-slave-l4q5"
server_key: "337bbe1e2c224734dee6c65708eef466"
server_key_iv: "5a2c37b6ba3644bffc5af7ab018e9e69"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.108556 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "6276b66869d94ef39f4a6e00209a812f"
format_stamp: "Formatted at 2026-09-23 22:16:31 on dist-test-slave-l4q5"
server_key: "337bbe1e2c224734dee6c65708eef466"
server_key_iv: "5a2c37b6ba3644bffc5af7ab018e9e69"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.109890 22216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:31.110648 22231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:31.110811 22216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:31.110901 22216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "6276b66869d94ef39f4a6e00209a812f"
format_stamp: "Formatted at 2026-09-23 22:16:31 on dist-test-slave-l4q5"
server_key: "337bbe1e2c224734dee6c65708eef466"
server_key_iv: "5a2c37b6ba3644bffc5af7ab018e9e69"
server_key_version: "encryptionkey@0"
I20260923 22:16:31.110965 22216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:31.121037 22216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:31.123379 22216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:31.123533 22216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:31.127576 22216 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44715
I20260923 22:16:31.127635 22283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44715 every 8 connection(s)
I20260923 22:16:31.127925 22216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:31.128892 22284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:31.131171 22284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: Bootstrap starting.
I20260923 22:16:31.131743 22284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:31.132001 22284 log.cc:824] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:31.132741 22284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: No bootstrap required, opened a new log
I20260923 22:16:31.134264 22284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260923 22:16:31.134403 22284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:31.134434 22284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6276b66869d94ef39f4a6e00209a812f, State: Initialized, Role: FOLLOWER
I20260923 22:16:31.134570 22284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260923 22:16:31.134629 22284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:31.134665 22284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:31.134711 22284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:31.135214 22284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260923 22:16:31.135305 22284 leader_election.cc:302] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6276b66869d94ef39f4a6e00209a812f; no voters:
I20260923 22:16:31.135469 22284 leader_election.cc:288] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:31.135533 22287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:31.135694 22287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [term 1 LEADER]: Becoming Leader. State: Replica: 6276b66869d94ef39f4a6e00209a812f, State: Running, Role: LEADER
I20260923 22:16:31.135720 22284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:31.135838 22287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260923 22:16:31.136204 22289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6276b66869d94ef39f4a6e00209a812f. Latest consensus state: current_term: 1 leader_uuid: "6276b66869d94ef39f4a6e00209a812f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } } }
I20260923 22:16:31.136282 22289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:31.136173 22288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6276b66869d94ef39f4a6e00209a812f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6276b66869d94ef39f4a6e00209a812f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } } }
I20260923 22:16:31.136420 22288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:31.137626 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22216
W20260923 22:16:31.137681 22303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:31.137722 22303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:31.137744 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:31.137800 22291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:31.137924 22291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:31.137946 1554 external_mini_cluster.cc:1428] Setting key 1951943406086d1ef4ccec7d22c4de4c
I20260923 22:16:31.139405 22291 catalog_manager.cc:1347] Generated new cluster ID: 145914dbc9ff4b75b8457c0c96b2f6e5
I20260923 22:16:31.139459 22291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:31.147960 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:31.138401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59773 (local address 127.0.0.1:44715)
0923 22:16:31.138550 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:31.138559 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:31.138593 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:16:31.138704 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:31.138710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:31.138761 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:31.138872 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:31.139200 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:31.140619 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:31.141668 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:31.142312 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:31.142427 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:31.147165 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:31.147170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:31.147170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:31.147405 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:31.147452 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:31.147470 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:31.147547 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:31.147627 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:16:31.147711 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260923 22:16:31.148663 22291 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:31.148917 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:31.149324 22291 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:31.154063 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:31.150349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49929 (local address 127.0.0.1:44715)
0923 22:16:31.150416 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:31.150418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:31.150523 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:16:31.150587 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:31.150588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:31.150598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:31.150644 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:31.150807 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:31.151546 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:31.152641 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:31.152954 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:31.153011 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:31.153706 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:31.153708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:31.153708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:31.153713 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:31.153729 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:31.153733 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:31.153882 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:31.153948 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:16:31.153980 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:16:31.157440 22291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6276b66869d94ef39f4a6e00209a812f: Generated new TSK 0
I20260923 22:16:31.157616 22291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:31.160764 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22216
2026-09-23T22:16:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:31 dist-test-slave-l4q5 krb5kdc[22315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:31 dist-test-slave-l4q5 krb5kdc[22315](info): set up 2 sockets
Sep 23 22:16:31 dist-test-slave-l4q5 krb5kdc[22315](info): commencing operation
krb5kdc: starting...
W20260923 22:16:32.115999 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:32 dist-test-slave-l4q5 krb5kdc[22315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:32.240899 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:32.321033 22328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:32.321267 22328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:32.321291 22328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:32.322497 22328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:32.322538 22328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:32.322551 22328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:32.322563 22328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:32.322574 22328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:32.324066 22328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:32.324333 22328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:32.324555 22328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:32.329448 22334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:32.329463 22336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:32.329463 22333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:32.329785 22328 server_base.cc:1034] running on GCE node
I20260923 22:16:32.330036 22328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:32.330339 22328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:32.331511 22328 hybrid_clock.cc:648] HybridClock initialized: now 1790201792331495 us; error 30 us; skew 500 ppm
Sep 23 22:16:32 dist-test-slave-l4q5 krb5kdc[22315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:32.334096 22328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:32.334511 22328 webserver.cc:460] Webserver started at http://127.0.0.1:36797/ using document root <none> and password file <none>
I20260923 22:16:32.334738 22328 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:32.334786 22328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:32.334909 22328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:32.335988 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "698fc075e8cf46a39f4bb44a359623ac"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-l4q5"
server_key: "372a0c16f972f7d396f68fc7483b4d4e"
server_key_iv: "feb585acefff9cd73cc280521b5e405a"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.336372 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "698fc075e8cf46a39f4bb44a359623ac"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-l4q5"
server_key: "372a0c16f972f7d396f68fc7483b4d4e"
server_key_iv: "feb585acefff9cd73cc280521b5e405a"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.337678 22328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:32.338455 22343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:32.338620 22328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:32.338702 22328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "698fc075e8cf46a39f4bb44a359623ac"
format_stamp: "Formatted at 2026-09-23 22:16:32 on dist-test-slave-l4q5"
server_key: "372a0c16f972f7d396f68fc7483b4d4e"
server_key_iv: "feb585acefff9cd73cc280521b5e405a"
server_key_version: "encryptionkey@0"
I20260923 22:16:32.338762 22328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:32.342171 22328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:32.344296 22328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:32.344424 22328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:32.348138 22328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42491
I20260923 22:16:32.348174 22395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42491 every 8 connection(s)
I20260923 22:16:32.348500 22328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:32.349435 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:32.351763 22396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac: Bootstrap starting.
I20260923 22:16:32.352409 22396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:32.352663 22396 log.cc:824] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:32.353520 22396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac: No bootstrap required, opened a new log
I20260923 22:16:32.355158 22396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260923 22:16:32.355309 22396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:32.355329 22396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698fc075e8cf46a39f4bb44a359623ac, State: Initialized, Role: FOLLOWER
I20260923 22:16:32.355451 22396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260923 22:16:32.355530 22396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:32.355557 22396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:32.355584 22396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:32.355783 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22328
I20260923 22:16:32.355883 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:32.356036 1554 external_mini_cluster.cc:1428] Setting key 1d00263cd358ddf9bcdca5ed62116764
I20260923 22:16:32.356467 22396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260923 22:16:32.356642 22396 leader_election.cc:302] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 698fc075e8cf46a39f4bb44a359623ac; no voters:
I20260923 22:16:32.356832 22396 leader_election.cc:288] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:32.356851 22401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:32.357066 22401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [term 1 LEADER]: Becoming Leader. State: Replica: 698fc075e8cf46a39f4bb44a359623ac, State: Running, Role: LEADER
I20260923 22:16:32.357088 22396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:32.357198 22401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260923 22:16:32.357545 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 698fc075e8cf46a39f4bb44a359623ac. Latest consensus state: current_term: 1 leader_uuid: "698fc075e8cf46a39f4bb44a359623ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } } }
I20260923 22:16:32.357654 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:32.357559 22402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "698fc075e8cf46a39f4bb44a359623ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698fc075e8cf46a39f4bb44a359623ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } } }
I20260923 22:16:32.357883 22402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:32.358058 22406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:32.358511 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:32.356656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41975 (local address 127.0.0.1:42491)
0923 22:16:32.356836 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:32.356846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:32.356879 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:16:32.357018 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:32.357027 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:32.357063 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:32.357238 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:32.357973 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:32.357977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:32.357977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:32.357982 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:32.358027 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:32.358042 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:32.358245 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:32.358276 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:16:32.358317 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260923 22:16:32.358543 22406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:32.359380 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:32.360525 22406 catalog_manager.cc:1347] Generated new cluster ID: 8bd1e61d73154a73876011fc8deb6846
I20260923 22:16:32.360579 22406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:32.361904 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:32.360800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44453 (local address 127.0.0.1:42491)
0923 22:16:32.360835 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:32.360836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:32.360936 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:16:32.360991 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:32.360992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:32.360997 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:32.361039 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:32.361612 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:32.361613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:32.361613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:32.361617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:32.361631 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:32.361635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:32.361682 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:32.361833 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:16:32.361845 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:32.379918 22406 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:32.380115 22406 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:32.393682 22406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 698fc075e8cf46a39f4bb44a359623ac: Generated new TSK 0
I20260923 22:16:32.393888 22406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:32.405637 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22328
2026-09-23T22:16:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:32 dist-test-slave-l4q5 krb5kdc[22427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:32 dist-test-slave-l4q5 krb5kdc[22427](info): set up 2 sockets
Sep 23 22:16:32 dist-test-slave-l4q5 krb5kdc[22427](info): commencing operation
krb5kdc: starting...
W20260923 22:16:33.372030 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:33.492125 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:33.571408 22440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:33.571622 22440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:33.571645 22440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:33.572857 22440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:33.572904 22440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:33.572916 22440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:33.572927 22440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:33.572938 22440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:33.574395 22440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:33.574618 22440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:33.574818 22440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:33.579540 22448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:33.579576 22445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:33.579721 22440 server_base.cc:1034] running on GCE node
W20260923 22:16:33.579824 22446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:33.580034 22440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:33.580333 22440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:33.581509 22440 hybrid_clock.cc:648] HybridClock initialized: now 1790201793581497 us; error 26 us; skew 500 ppm
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:33.583907 22440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:33.584436 22440 webserver.cc:460] Webserver started at http://127.0.0.1:43889/ using document root <none> and password file <none>
I20260923 22:16:33.584671 22440 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:33.584729 22440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:33.584864 22440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:33.585942 22440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "8301ca1e91bd4656a12f38f2ba9c663b"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-l4q5"
server_key: "f7a2b15127b41ce5a4b716b0b17777ca"
server_key_iv: "2a91c7cac8ad782283e9927cdcbe2b4f"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.586267 22440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "8301ca1e91bd4656a12f38f2ba9c663b"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-l4q5"
server_key: "f7a2b15127b41ce5a4b716b0b17777ca"
server_key_iv: "2a91c7cac8ad782283e9927cdcbe2b4f"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.587615 22440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:33.588490 22455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:33.588650 22440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:33.588732 22440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "8301ca1e91bd4656a12f38f2ba9c663b"
format_stamp: "Formatted at 2026-09-23 22:16:33 on dist-test-slave-l4q5"
server_key: "f7a2b15127b41ce5a4b716b0b17777ca"
server_key_iv: "2a91c7cac8ad782283e9927cdcbe2b4f"
server_key_version: "encryptionkey@0"
I20260923 22:16:33.588795 22440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:33.608543 22440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:33.610929 22440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:33.611068 22440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:33.614946 22440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36505
I20260923 22:16:33.614979 22507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36505 every 8 connection(s)
I20260923 22:16:33.615310 22440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:33.616276 22508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:33.616940 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22440
I20260923 22:16:33.617041 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:33.617199 1554 external_mini_cluster.cc:1428] Setting key dd889b7b0d9e36cf8e9d3c9a9b5d5de0
I20260923 22:16:33.619213 22508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b: Bootstrap starting.
I20260923 22:16:33.619903 22508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:33.620230 22508 log.cc:824] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:33.620893 22508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b: No bootstrap required, opened a new log
I20260923 22:16:33.622941 22508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } }
I20260923 22:16:33.623081 22508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:33.623114 22508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8301ca1e91bd4656a12f38f2ba9c663b, State: Initialized, Role: FOLLOWER
I20260923 22:16:33.623239 22508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } }
I20260923 22:16:33.623310 22508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:33.623342 22508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:33.623435 22508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:33.624141 22508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } }
I20260923 22:16:33.624303 22508 leader_election.cc:302] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8301ca1e91bd4656a12f38f2ba9c663b; no voters:
I20260923 22:16:33.624473 22508 leader_election.cc:288] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:33.624503 22513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:33.624614 22513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [term 1 LEADER]: Becoming Leader. State: Replica: 8301ca1e91bd4656a12f38f2ba9c663b, State: Running, Role: LEADER
I20260923 22:16:33.624758 22513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } }
I20260923 22:16:33.624758 22508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:33.625180 22514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } } }
I20260923 22:16:33.625262 22514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:33.625488 22517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:33.625454 22515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8301ca1e91bd4656a12f38f2ba9c663b. Latest consensus state: current_term: 1 leader_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8301ca1e91bd4656a12f38f2ba9c663b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36505 } } }
I20260923 22:16:33.625818 22515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:33.626497 22517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:33.628021 22517 catalog_manager.cc:1347] Generated new cluster ID: 76fd5a88a6a74fba9353093c74c4d016
I20260923 22:16:33.628067 22517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:33.629153 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:33.617748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:49033 (local address 127.0.0.1:36505)
0923 22:16:33.617930 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:33.617940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:33.617973 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:16:33.618483 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:33.618492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:33.618563 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:33.618710 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:33.619271 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.620583 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.621349 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.621966 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.622090 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:33.624740 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:33.624751 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:33.624751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:33.624764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:33.627316 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:33.627792 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:33.627796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:33.627796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:33.627906 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:33.628156 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:33.628157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:33.628157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:33.628484 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:33.628653 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:33.628847 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 22:16:33.628952 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260923 22:16:33.630101 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:33.638856 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:33.631591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:52369 (local address 127.0.0.1:36505)
0923 22:16:33.631630 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:33.631632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:33.631640 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:33.631660 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:33.631661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:33.631670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:33.631722 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:33.632403 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.633141 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.634182 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:33.634516 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:33.634570 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:33.636042 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:33.636043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:33.636043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:33.636050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:33.637516 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:33.637947 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:33.637948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:33.637949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:33.638007 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:33.638233 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:33.638233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:33.638234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:33.638356 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:33.638449 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:33.638691 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:16:33.638759 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:33.641489 22517 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:33.642004 22517 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:33.648265 22517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8301ca1e91bd4656a12f38f2ba9c663b: Generated new TSK 0
I20260923 22:16:33.648483 22517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:33.652588 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22440
W20260923 22:16:33.656710 22425 connection.cc:535] client connection to 127.0.0.1:36505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36505): Connection reset by peer (error 104)
2026-09-23T22:16:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1247 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22539](info): set up 2 sockets
Sep 23 22:16:33 dist-test-slave-l4q5 krb5kdc[22539](info): commencing operation
krb5kdc: starting...
W20260923 22:16:34.612026 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:34 dist-test-slave-l4q5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:34.725329 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:34.806878 22552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:34.807083 22552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:34.807114 22552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:34.808285 22552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:34.808328 22552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:34.808346 22552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:34.808362 22552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:34.808377 22552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:34.809851 22552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:34.810101 22552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:34.810359 22552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:34.815225 22560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:34.815236 22552 server_base.cc:1034] running on GCE node
W20260923 22:16:34.815220 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:34.815220 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:34.815776 22552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:34.816073 22552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:34.817230 22552 hybrid_clock.cc:648] HybridClock initialized: now 1790201794817209 us; error 42 us; skew 500 ppm
Sep 23 22:16:34 dist-test-slave-l4q5 krb5kdc[22539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:34.819628 22552 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:34.820050 22552 webserver.cc:460] Webserver started at http://127.0.0.1:36379/ using document root <none> and password file <none>
I20260923 22:16:34.820287 22552 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:34.820336 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:34.820457 22552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:34.821472 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "ad01fc1a530b480098c7660e2d48d2fd"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-l4q5"
server_key: "16e60a0fe6d177e51a964d1879e8660e"
server_key_iv: "c0d80381701a5c21e660e905439cfe98"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.821789 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "ad01fc1a530b480098c7660e2d48d2fd"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-l4q5"
server_key: "16e60a0fe6d177e51a964d1879e8660e"
server_key_iv: "c0d80381701a5c21e660e905439cfe98"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.823014 22552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:34.823757 22567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:34.823971 22552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:34.824048 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "ad01fc1a530b480098c7660e2d48d2fd"
format_stamp: "Formatted at 2026-09-23 22:16:34 on dist-test-slave-l4q5"
server_key: "16e60a0fe6d177e51a964d1879e8660e"
server_key_iv: "c0d80381701a5c21e660e905439cfe98"
server_key_version: "encryptionkey@0"
I20260923 22:16:34.824112 22552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:34.838958 22552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:34.841549 22552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:34.841701 22552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:34.845862 22552 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40709
I20260923 22:16:34.845953 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40709 every 8 connection(s)
I20260923 22:16:34.846376 22552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:34.847393 22620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:34.849761 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd: Bootstrap starting.
I20260923 22:16:34.849999 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22552
I20260923 22:16:34.850102 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:34.850260 1554 external_mini_cluster.cc:1428] Setting key 3ccc2025ccfb5dcf30bc673253c24c24
I20260923 22:16:34.850359 22620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:34.850631 22620 log.cc:824] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:34.851333 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd: No bootstrap required, opened a new log
I20260923 22:16:34.853641 22620 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260923 22:16:34.853830 22620 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:34.853861 22620 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad01fc1a530b480098c7660e2d48d2fd, State: Initialized, Role: FOLLOWER
I20260923 22:16:34.853984 22620 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260923 22:16:34.854066 22620 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:34.854100 22620 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:34.854151 22620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:34.854786 22620 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260923 22:16:34.854916 22620 leader_election.cc:302] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad01fc1a530b480098c7660e2d48d2fd; no voters:
I20260923 22:16:34.855118 22620 leader_election.cc:288] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:34.855301 22625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:34.855389 22620 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:34.855495 22625 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [term 1 LEADER]: Becoming Leader. State: Replica: ad01fc1a530b480098c7660e2d48d2fd, State: Running, Role: LEADER
I20260923 22:16:34.855643 22625 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } }
I20260923 22:16:34.856030 22626 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad01fc1a530b480098c7660e2d48d2fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } } }
I20260923 22:16:34.856120 22626 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:34.856315 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad01fc1a530b480098c7660e2d48d2fd. Latest consensus state: current_term: 1 leader_uuid: "ad01fc1a530b480098c7660e2d48d2fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad01fc1a530b480098c7660e2d48d2fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40709 } } }
I20260923 22:16:34.856351 22634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:34.856379 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:34.856364 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:34.850783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:39205 (local address 127.0.0.1:40709)
0923 22:16:34.850953 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:34.850960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:34.850985 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:16:34.851533 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:34.851541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:34.851596 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:34.851723 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:34.851994 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:34.852980 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:34.854061 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:34.854575 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:34.854666 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:34.855473 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:34.855476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:34.855477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:34.855677 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:34.855719 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:34.855735 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:34.855811 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:34.856102 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 22:16:34.856132 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260923 22:16:34.856971 22634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:34.857607 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:34.858582 22634 catalog_manager.cc:1347] Generated new cluster ID: 6f99e8bcbedb4469ae7bcb3093258c9e
I20260923 22:16:34.858634 22634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:34.862123 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:34.859010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:49147 (local address 127.0.0.1:40709)
0923 22:16:34.859044 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:34.859046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:34.859154 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:16:34.859214 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:34.859215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:34.859225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:34.859266 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:34.859392 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:34.859873 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:34.860798 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:34.861117 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:34.861184 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:34.861883 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:34.861884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:34.861885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:34.861889 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:34.861903 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:34.861907 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:34.861956 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:34.862034 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:16:34.862047 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:34.871461 22634 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:34.872006 22634 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:34.884033 22634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad01fc1a530b480098c7660e2d48d2fd: Generated new TSK 0
I20260923 22:16:34.884258 22634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:34.891611 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22552
W20260923 22:16:34.895546 22537 connection.cc:535] client connection to 127.0.0.1:40709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40709): Connection reset by peer (error 104)
2026-09-23T22:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:34 dist-test-slave-l4q5 krb5kdc[22651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:34 dist-test-slave-l4q5 krb5kdc[22651](info): set up 2 sockets
Sep 23 22:16:34 dist-test-slave-l4q5 krb5kdc[22651](info): commencing operation
krb5kdc: starting...
W20260923 22:16:35.860123 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:35 dist-test-slave-l4q5 krb5kdc[22651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:35.985473 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:36.069465 22664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:36.069698 22664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:36.069723 22664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:36.070890 22664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:36.070931 22664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:36.070945 22664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:36.070955 22664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:36.070966 22664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:36.072499 22664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:36.072715 22664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:36.072911 22664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:36.077998 22669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:36.078141 22664 server_base.cc:1034] running on GCE node
W20260923 22:16:36.078130 22672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:36.078014 22670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:36.078509 22664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:36.078828 22664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:36.079994 22664 hybrid_clock.cc:648] HybridClock initialized: now 1790201796079966 us; error 46 us; skew 500 ppm
Sep 23 22:16:36 dist-test-slave-l4q5 krb5kdc[22651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201796, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:36.082459 22664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:36.082911 22664 webserver.cc:460] Webserver started at http://127.0.0.1:40607/ using document root <none> and password file <none>
I20260923 22:16:36.083110 22664 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:36.083158 22664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:36.083283 22664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:36.084395 22664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "9c294c41c75f44999d14b135dc9b24a5"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-l4q5"
server_key: "04f93c54dd9e9fab431d9cb1666baacf"
server_key_iv: "c90c5c6f1e793443126914841dcfb3cf"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.084765 22664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "9c294c41c75f44999d14b135dc9b24a5"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-l4q5"
server_key: "04f93c54dd9e9fab431d9cb1666baacf"
server_key_iv: "c90c5c6f1e793443126914841dcfb3cf"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.085991 22664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:36.086652 22679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:36.086848 22664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:36.086946 22664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "9c294c41c75f44999d14b135dc9b24a5"
format_stamp: "Formatted at 2026-09-23 22:16:36 on dist-test-slave-l4q5"
server_key: "04f93c54dd9e9fab431d9cb1666baacf"
server_key_iv: "c90c5c6f1e793443126914841dcfb3cf"
server_key_version: "encryptionkey@0"
I20260923 22:16:36.087010 22664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:36.090358 22664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:36.092415 22664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:36.092540 22664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:36.096203 22664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37831
I20260923 22:16:36.096267 22731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37831 every 8 connection(s)
I20260923 22:16:36.096577 22664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:36.097504 22732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:36.099730 22732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5: Bootstrap starting.
I20260923 22:16:36.100288 22732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:36.100548 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22664
I20260923 22:16:36.100577 22732 log.cc:824] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:36.100663 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:36.100844 1554 external_mini_cluster.cc:1428] Setting key 2ed3167ef7b4b5816937b69b4c4180e5
I20260923 22:16:36.101318 22732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5: No bootstrap required, opened a new log
I20260923 22:16:36.102974 22732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } }
I20260923 22:16:36.103116 22732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:36.103148 22732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c294c41c75f44999d14b135dc9b24a5, State: Initialized, Role: FOLLOWER
I20260923 22:16:36.103318 22732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } }
I20260923 22:16:36.103426 22732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:36.103469 22732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:36.103523 22732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:36.104396 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:36.101327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:51107 (local address 127.0.0.1:37831)
0923 22:16:36.101489 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:36.101497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:36.101521 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:16:36.102965 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:36.102973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:36.103005 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:36.103147 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:36.103839 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:36.103842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:36.103843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:36.103846 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:36.103908 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:36.103920 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:36.103981 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:36.104066 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:16:36.104110 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260923 22:16:36.104637 22732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } }
I20260923 22:16:36.104748 22732 leader_election.cc:302] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c294c41c75f44999d14b135dc9b24a5; no voters:
I20260923 22:16:36.104908 22732 leader_election.cc:288] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:36.104935 22737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:36.105068 22737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9c294c41c75f44999d14b135dc9b24a5, State: Running, Role: LEADER
I20260923 22:16:36.105201 22732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:36.105242 22737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } }
I20260923 22:16:36.105290 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:36.105767 22738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c294c41c75f44999d14b135dc9b24a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } } }
I20260923 22:16:36.105857 22738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:36.106199 22739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c294c41c75f44999d14b135dc9b24a5. Latest consensus state: current_term: 1 leader_uuid: "9c294c41c75f44999d14b135dc9b24a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c294c41c75f44999d14b135dc9b24a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37831 } } }
I20260923 22:16:36.106650 22739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:36.106727 22744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:36.107232 22744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:36.108017 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:36.106852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:35437 (local address 127.0.0.1:37831)
0923 22:16:36.106896 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:36.106899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:36.107033 (+ 134us) server_negotiation.cc:407] Connection header received
0923 22:16:36.107111 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:36.107112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:36.107119 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:36.107169 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:36.107772 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:36.107773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:36.107773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:36.107777 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:36.107795 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:36.107800 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:36.107852 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:36.107943 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:16:36.107956 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:36.109571 22744 catalog_manager.cc:1347] Generated new cluster ID: f84f7aa0d44140ba9e65276a31295363
I20260923 22:16:36.109616 22744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:36.127038 22744 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:36.127218 22744 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:36.136435 22744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c294c41c75f44999d14b135dc9b24a5: Generated new TSK 0
I20260923 22:16:36.136649 22744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:36.154961 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22664
2026-09-23T22:16:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:36 dist-test-slave-l4q5 krb5kdc[22764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:36 dist-test-slave-l4q5 krb5kdc[22764](info): set up 2 sockets
Sep 23 22:16:36 dist-test-slave-l4q5 krb5kdc[22764](info): commencing operation
krb5kdc: starting...
W20260923 22:16:37.112111 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:37.236317 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:37.314868 22777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:37.315095 22777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:37.315129 22777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:37.316376 22777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:37.316421 22777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:37.316439 22777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:37.316455 22777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:37.316469 22777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:37.317979 22777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:37.318233 22777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:37.318444 22777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:37.323166 22783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:37.323168 22785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:37.323167 22782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:37.323531 22777 server_base.cc:1034] running on GCE node
I20260923 22:16:37.323799 22777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:37.324049 22777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:37.325223 22777 hybrid_clock.cc:648] HybridClock initialized: now 1790201797325198 us; error 33 us; skew 500 ppm
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:37.327641 22777 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:37.328042 22777 webserver.cc:460] Webserver started at http://127.0.0.1:43055/ using document root <none> and password file <none>
I20260923 22:16:37.328270 22777 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:37.328320 22777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:37.328406 22777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:37.329391 22777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "44849b78c9d8428a96b6a527b012c9bd"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-l4q5"
server_key: "80962cd59794d06a96abda14cd7580bb"
server_key_iv: "d21290f28187c1e7e72c83d05a13e5c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.329663 22777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "44849b78c9d8428a96b6a527b012c9bd"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-l4q5"
server_key: "80962cd59794d06a96abda14cd7580bb"
server_key_iv: "d21290f28187c1e7e72c83d05a13e5c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.330873 22777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:37.331593 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:37.331768 22777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:37.331852 22777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "44849b78c9d8428a96b6a527b012c9bd"
format_stamp: "Formatted at 2026-09-23 22:16:37 on dist-test-slave-l4q5"
server_key: "80962cd59794d06a96abda14cd7580bb"
server_key_iv: "d21290f28187c1e7e72c83d05a13e5c9"
server_key_version: "encryptionkey@0"
I20260923 22:16:37.331907 22777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:37.343459 22777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:37.345577 22777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:37.345687 22777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:37.349377 22777 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34421
I20260923 22:16:37.349412 22844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34421 every 8 connection(s)
I20260923 22:16:37.349731 22777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:37.350739 22845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:37.350848 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22777
I20260923 22:16:37.350947 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:37.351145 1554 external_mini_cluster.cc:1428] Setting key aabc06ffbdbefa40bc81f03ee75faa91
I20260923 22:16:37.354231 22845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: Bootstrap starting.
I20260923 22:16:37.354929 22845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:37.355211 22845 log.cc:824] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:37.355886 22845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: No bootstrap required, opened a new log
I20260923 22:16:37.358063 22845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260923 22:16:37.358253 22845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:37.358289 22845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44849b78c9d8428a96b6a527b012c9bd, State: Initialized, Role: FOLLOWER
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:16:37.358438 22845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260923 22:16:37.358520 22845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:37.358551 22845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:37.358599 22845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:37.359277 22845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260923 22:16:37.359390 22845 leader_election.cc:302] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44849b78c9d8428a96b6a527b012c9bd; no voters:
I20260923 22:16:37.359561 22845 leader_election.cc:288] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:37.359651 22850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:37.359794 22850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [term 1 LEADER]: Becoming Leader. State: Replica: 44849b78c9d8428a96b6a527b012c9bd, State: Running, Role: LEADER
I20260923 22:16:37.359840 22845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:37.359889 22850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260923 22:16:37.360219 22850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44849b78c9d8428a96b6a527b012c9bd. Latest consensus state: current_term: 1 leader_uuid: "44849b78c9d8428a96b6a527b012c9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260923 22:16:37.360352 22850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:37.361222 22851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44849b78c9d8428a96b6a527b012c9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44849b78c9d8428a96b6a527b012c9bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260923 22:16:37.361425 22851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:37.361558 22866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:37.361609 22866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:37.361658 22858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:37.361827 22858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:37.362618 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:37.351658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:51965 (local address 127.0.0.1:34421)
0923 22:16:37.351857 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:37.351867 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:37.351897 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:16:37.352372 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:37.352381 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:37.352448 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:37.352592 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:37.353458 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.354783 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.355642 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.356270 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.356389 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:37.358841 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:37.358845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:37.358846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:37.358857 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:37.360907 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:37.361375 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:37.361378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:37.361378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:37.361478 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:37.361738 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:37.361739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:37.361739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:37.362012 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:37.362131 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:37.362308 (+ 177us) server_negotiation.cc:299] Negotiation successful
0923 22:16:37.362429 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":64,"threads_started":1}
I20260923 22:16:37.363406 22858 catalog_manager.cc:1347] Generated new cluster ID: 9a1272837e4a454d97c510f9faf0bcf4
I20260923 22:16:37.363453 22858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:37.364593 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:37.375321 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:37.366129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:49299 (local address 127.0.0.1:34421)
0923 22:16:37.368256 (+ 2127us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:37.368258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:37.368266 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:16:37.368287 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:37.368288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:37.368297 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:37.368349 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:37.369035 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.369769 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.370766 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:37.371088 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:37.371143 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:37.372644 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:37.372646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:37.372646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:37.372654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:37.374121 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:37.374505 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:37.374506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:37.374507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:37.374567 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:37.374766 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:37.374767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:37.374768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:37.374899 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:37.375000 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:37.375156 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:16:37.375223 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2104}
I20260923 22:16:37.375324 22858 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:37.376094 22858 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:37.381642 22858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44849b78c9d8428a96b6a527b012c9bd: Generated new TSK 0
I20260923 22:16:37.381845 22858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:37.386698 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22777
W20260923 22:16:37.390400 22762 connection.cc:535] client connection to 127.0.0.1:34421 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34421): Connection reset by peer (error 104)
2026-09-23T22:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22876](info): set up 2 sockets
Sep 23 22:16:37 dist-test-slave-l4q5 krb5kdc[22876](info): commencing operation
krb5kdc: starting...
W20260923 22:16:38.344004 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:38 dist-test-slave-l4q5 krb5kdc[22876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:38.464148 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:38.543572 22889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:38.543792 22889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:38.543824 22889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:38.545034 22889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:38.545078 22889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:38.545097 22889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:38.545113 22889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:38.545128 22889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:38.546619 22889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:38.546864 22889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:38.547067 22889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:38.551617 22895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:38.551626 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:38.551633 22897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:38.552017 22889 server_base.cc:1034] running on GCE node
I20260923 22:16:38.552292 22889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:38.552567 22889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:38.553717 22889 hybrid_clock.cc:648] HybridClock initialized: now 1790201798553691 us; error 44 us; skew 500 ppm
Sep 23 22:16:38 dist-test-slave-l4q5 krb5kdc[22876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:38.556211 22889 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:38.556766 22889 webserver.cc:460] Webserver started at http://127.0.0.1:36869/ using document root <none> and password file <none>
I20260923 22:16:38.556972 22889 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:38.557010 22889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:38.557099 22889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:38.558112 22889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "dcf435817f1c4a91b4cc23c3498c95ee"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-l4q5"
server_key: "92c4228f5d72b583c104dc8ab4355aed"
server_key_iv: "d9bef6912b61146c063dd8a47d32155c"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.558430 22889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "dcf435817f1c4a91b4cc23c3498c95ee"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-l4q5"
server_key: "92c4228f5d72b583c104dc8ab4355aed"
server_key_iv: "d9bef6912b61146c063dd8a47d32155c"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.559706 22889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:38.560470 22904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:38.560608 22889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:38.560678 22889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "dcf435817f1c4a91b4cc23c3498c95ee"
format_stamp: "Formatted at 2026-09-23 22:16:38 on dist-test-slave-l4q5"
server_key: "92c4228f5d72b583c104dc8ab4355aed"
server_key_iv: "d9bef6912b61146c063dd8a47d32155c"
server_key_version: "encryptionkey@0"
I20260923 22:16:38.560740 22889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:38.572031 22889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:38.574054 22889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:38.574164 22889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:38.577857 22889 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35405
I20260923 22:16:38.577900 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35405 every 8 connection(s)
I20260923 22:16:38.578215 22889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:38.578858 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 22889
I20260923 22:16:38.578970 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:38.579155 1554 external_mini_cluster.cc:1428] Setting key b8ee08a577589fa9eb2ef6a09e1f70c7
I20260923 22:16:38.579159 22957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:38.582587 22957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: Bootstrap starting.
I20260923 22:16:38.583214 22957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:38.583492 22957 log.cc:824] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:38.584270 22957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: No bootstrap required, opened a new log
I20260923 22:16:38.585055 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:38.579663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:49251 (local address 127.0.0.1:35405)
0923 22:16:38.579883 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:38.579892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:38.579924 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:16:38.580413 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:38.580421 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:38.580491 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:38.580645 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:38.580859 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:38.582178 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:38.582975 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:38.583566 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:38.583726 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:38.584425 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:38.584429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:38.584430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:38.584591 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:38.584622 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:38.584632 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:38.584697 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:38.584807 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:16:38.584831 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260923 22:16:38.586074 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:38.586516 22957 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260923 22:16:38.586694 22957 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:38.586737 22957 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf435817f1c4a91b4cc23c3498c95ee, State: Initialized, Role: FOLLOWER
I20260923 22:16:38.586860 22957 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260923 22:16:38.586934 22957 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:38.586966 22957 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:38.587013 22957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:38.587752 22957 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260923 22:16:38.587863 22957 leader_election.cc:302] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf435817f1c4a91b4cc23c3498c95ee; no voters:
I20260923 22:16:38.588025 22957 leader_election.cc:288] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:38.588061 22967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:38.588244 22967 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [term 1 LEADER]: Becoming Leader. State: Replica: dcf435817f1c4a91b4cc23c3498c95ee, State: Running, Role: LEADER
I20260923 22:16:38.588284 22957 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:38.588410 22967 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260923 22:16:38.589648 22969 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf435817f1c4a91b4cc23c3498c95ee. Latest consensus state: current_term: 1 leader_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260923 22:16:38.589802 22969 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:38.589833 22968 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf435817f1c4a91b4cc23c3498c95ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260923 22:16:38.589910 22968 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:38.590787 22981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:38.590847 22981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:38.590948 22983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:38.591115 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:38.587546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:51585 (local address 127.0.0.1:35405)
0923 22:16:38.587583 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:38.587584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:38.587617 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:16:38.587686 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:38.587687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:38.587694 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:38.587741 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:38.587881 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:38.588634 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:38.589777 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:38.590054 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:38.590125 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:38.590820 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:38.590822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:38.590826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:38.590830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:38.590845 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:38.590849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:38.590910 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:38.591051 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:16:38.591061 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:16:38.591226 22983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:38.592693 22983 catalog_manager.cc:1347] Generated new cluster ID: 6884a04bd7904d58b9382682772566a4
I20260923 22:16:38.592744 22983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:38.608294 22983 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:38.608982 22983 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:38.617209 22983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf435817f1c4a91b4cc23c3498c95ee: Generated new TSK 0
I20260923 22:16:38.617393 22983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:38.625243 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 22889
W20260923 22:16:38.629004 22874 connection.cc:535] client connection to 127.0.0.1:35405 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35405): Connection reset by peer (error 104)
2026-09-23T22:16:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:38 dist-test-slave-l4q5 krb5kdc[22988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:38 dist-test-slave-l4q5 krb5kdc[22988](info): set up 2 sockets
Sep 23 22:16:38 dist-test-slave-l4q5 krb5kdc[22988](info): commencing operation
krb5kdc: starting...
W20260923 22:16:39.600088 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:39 dist-test-slave-l4q5 krb5kdc[22988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:16:39.718762 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:39.797575 23001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:39.797812 23001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:39.797847 23001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:39.799075 23001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:39.799135 23001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:39.799153 23001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:39.799170 23001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:39.799185 23001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:39.800797 23001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:39.801064 23001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:39.801292 23001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:39.806221 23009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:39.806293 23006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:39.806383 23007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:39.806445 23001 server_base.cc:1034] running on GCE node
I20260923 22:16:39.806679 23001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:39.806960 23001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:39.808138 23001 hybrid_clock.cc:648] HybridClock initialized: now 1790201799808113 us; error 43 us; skew 500 ppm
Sep 23 22:16:39 dist-test-slave-l4q5 krb5kdc[22988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201799, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:39.810910 23001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:16:39.811426 23001 webserver.cc:460] Webserver started at http://127.0.0.1:40497/ using document root <none> and password file <none>
I20260923 22:16:39.811658 23001 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:39.811714 23001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:39.811846 23001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:39.812937 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "81d0392af01b4a8fbe9bab07b10ecfe8"
format_stamp: "Formatted at 2026-09-23 22:16:39 on dist-test-slave-l4q5"
server_key: "126f5c871f33a1152280880226028b6e"
server_key_iv: "96f56856d07eb8fee8106f52ef4843da"
server_key_version: "encryptionkey@0"
I20260923 22:16:39.813268 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "81d0392af01b4a8fbe9bab07b10ecfe8"
format_stamp: "Formatted at 2026-09-23 22:16:39 on dist-test-slave-l4q5"
server_key: "126f5c871f33a1152280880226028b6e"
server_key_iv: "96f56856d07eb8fee8106f52ef4843da"
server_key_version: "encryptionkey@0"
I20260923 22:16:39.814517 23001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:39.815331 23016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:39.815534 23001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:39.815622 23001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "81d0392af01b4a8fbe9bab07b10ecfe8"
format_stamp: "Formatted at 2026-09-23 22:16:39 on dist-test-slave-l4q5"
server_key: "126f5c871f33a1152280880226028b6e"
server_key_iv: "96f56856d07eb8fee8106f52ef4843da"
server_key_version: "encryptionkey@0"
I20260923 22:16:39.815685 23001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:39.819108 23001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:39.821249 23001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:39.821393 23001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:39.825277 23001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45941
I20260923 22:16:39.825330 23068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45941 every 8 connection(s)
I20260923 22:16:39.825659 23001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:39.826769 23069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:39.829125 23069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8: Bootstrap starting.
I20260923 22:16:39.829684 23069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:39.829926 23069 log.cc:824] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:39.830778 23069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8: No bootstrap required, opened a new log
I20260923 22:16:39.832507 23069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } }
I20260923 22:16:39.832664 23069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:39.832686 23069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d0392af01b4a8fbe9bab07b10ecfe8, State: Initialized, Role: FOLLOWER
I20260923 22:16:39.832798 23069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } }
I20260923 22:16:39.832854 23069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:39.832882 23069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:39.832922 23069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:39.833243 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23001
I20260923 22:16:39.833353 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:39.833532 1554 external_mini_cluster.cc:1428] Setting key 384576ad35198b3f08aaa2280c28a144
I20260923 22:16:39.833719 23069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } }
I20260923 22:16:39.833822 23069 leader_election.cc:302] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81d0392af01b4a8fbe9bab07b10ecfe8; no voters:
I20260923 22:16:39.834007 23069 leader_election.cc:288] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:39.834333 23069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:39.834400 23072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:39.834504 23072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [term 1 LEADER]: Becoming Leader. State: Replica: 81d0392af01b4a8fbe9bab07b10ecfe8, State: Running, Role: LEADER
I20260923 22:16:39.834635 23072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } }
I20260923 22:16:39.834993 23072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81d0392af01b4a8fbe9bab07b10ecfe8. Latest consensus state: current_term: 1 leader_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } } }
I20260923 22:16:39.835002 23076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d0392af01b4a8fbe9bab07b10ecfe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45941 } } }
I20260923 22:16:39.835103 23072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:39.835121 23076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:39.835323 23083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:39.835906 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:39.834060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:59031 (local address 127.0.0.1:45941)
0923 22:16:39.834305 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:39.834314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:39.834342 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:16:39.834453 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:39.834461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:39.834496 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:39.834640 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:39.835381 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:39.835385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:39.835385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:39.835390 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:39.835434 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:39.835449 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:39.835502 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:39.835661 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:16:39.835709 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
I20260923 22:16:39.836212 23083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:39.836937 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:39.837874 23083 catalog_manager.cc:1347] Generated new cluster ID: 209059cb7d784de48879d2150054daa2
I20260923 22:16:39.837924 23083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:39.839440 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:39.838390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:33607 (local address 127.0.0.1:45941)
0923 22:16:39.838420 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:39.838422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:39.838540 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:16:39.838623 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:39.838624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:39.838630 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:39.838694 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:39.839217 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:39.839218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:39.839218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:39.839223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:39.839239 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:39.839243 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:39.839304 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:39.839355 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:16:39.839369 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:16:39.845494 23083 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:39.845654 23083 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:39.852837 23083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81d0392af01b4a8fbe9bab07b10ecfe8: Generated new TSK 0
I20260923 22:16:39.853036 23083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:39.855863 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23001
2026-09-23T22:16:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1230 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:39 dist-test-slave-l4q5 krb5kdc[23100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:39 dist-test-slave-l4q5 krb5kdc[23100](info): set up 2 sockets
Sep 23 22:16:39 dist-test-slave-l4q5 krb5kdc[23100](info): commencing operation
krb5kdc: starting...
W20260923 22:16:40.808022 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:40 dist-test-slave-l4q5 krb5kdc[23100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:40.927408 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36821
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:41.007804 23113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:41.008023 23113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:41.008044 23113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:41.009299 23113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:41.009351 23113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:41.009364 23113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:41.009375 23113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:41.009387 23113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:41.011101 23113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:41.011337 23113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:41.011580 23113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:41.016932 23121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:41.016947 23118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:41.016994 23113 server_base.cc:1034] running on GCE node
W20260923 22:16:41.016919 23119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:41.017365 23113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:41.017669 23113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:41.018862 23113 hybrid_clock.cc:648] HybridClock initialized: now 1790201801018845 us; error 36 us; skew 500 ppm
Sep 23 22:16:41 dist-test-slave-l4q5 krb5kdc[23100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201801, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:41.021675 23113 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:41.022231 23113 webserver.cc:460] Webserver started at http://127.1.132.190:37897/ using document root <none> and password file <none>
I20260923 22:16:41.022457 23113 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:41.022511 23113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:41.022753 23113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:41.023945 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "859cbac7e986458ba5e655aab54811e0"
format_stamp: "Formatted at 2026-09-23 22:16:41 on dist-test-slave-l4q5"
server_key: "4aa1219e0a3c378123ac70f20eb77118"
server_key_iv: "7020f041bf3dcd5b7083fa5d2c1ba0ed"
server_key_version: "encryptionkey@0"
I20260923 22:16:41.024331 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "859cbac7e986458ba5e655aab54811e0"
format_stamp: "Formatted at 2026-09-23 22:16:41 on dist-test-slave-l4q5"
server_key: "4aa1219e0a3c378123ac70f20eb77118"
server_key_iv: "7020f041bf3dcd5b7083fa5d2c1ba0ed"
server_key_version: "encryptionkey@0"
I20260923 22:16:41.025652 23113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:41.026437 23128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:41.026641 23113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:41.026731 23113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "859cbac7e986458ba5e655aab54811e0"
format_stamp: "Formatted at 2026-09-23 22:16:41 on dist-test-slave-l4q5"
server_key: "4aa1219e0a3c378123ac70f20eb77118"
server_key_iv: "7020f041bf3dcd5b7083fa5d2c1ba0ed"
server_key_version: "encryptionkey@0"
I20260923 22:16:41.026798 23113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:41.038709 23113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:41.040858 23113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:41.040997 23113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:41.044754 23113 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36821
I20260923 22:16:41.044806 23180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36821 every 8 connection(s)
I20260923 22:16:41.045120 23113 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:41.045956 23181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:41.048280 23181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0: Bootstrap starting.
I20260923 22:16:41.048851 23181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:41.049139 23181 log.cc:824] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:41.049819 23181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0: No bootstrap required, opened a new log
I20260923 22:16:41.051487 23181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } }
I20260923 22:16:41.051632 23181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:41.051654 23181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859cbac7e986458ba5e655aab54811e0, State: Initialized, Role: FOLLOWER
I20260923 22:16:41.051789 23181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } }
I20260923 22:16:41.051853 23181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:41.051875 23181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:41.051923 23181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:41.052130 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23113
I20260923 22:16:41.052244 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:41.052414 1554 external_mini_cluster.cc:1428] Setting key 608b0bb420161dab09865ad8249d5b32
I20260923 22:16:41.052915 23181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } }
I20260923 22:16:41.053082 23181 leader_election.cc:302] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 859cbac7e986458ba5e655aab54811e0; no voters:
I20260923 22:16:41.053224 23181 leader_election.cc:288] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:41.053252 23186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:41.053395 23186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [term 1 LEADER]: Becoming Leader. State: Replica: 859cbac7e986458ba5e655aab54811e0, State: Running, Role: LEADER
I20260923 22:16:41.053473 23181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:41.053485 23186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } }
I20260923 22:16:41.054052 23187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "859cbac7e986458ba5e655aab54811e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } } }
I20260923 22:16:41.054061 23188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 859cbac7e986458ba5e655aab54811e0. Latest consensus state: current_term: 1 leader_uuid: "859cbac7e986458ba5e655aab54811e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859cbac7e986458ba5e655aab54811e0" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36821 } } }
I20260923 22:16:41.054203 23187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:41.054214 23188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:41.054443 23195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:41.054983 23195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:41.056856 23195 catalog_manager.cc:1347] Generated new cluster ID: bff0977e1db84a31920945f0bdb55b11
I20260923 22:16:41.056905 23195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:16:41 dist-test-slave-l4q5 krb5kdc[23100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:41.062364 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.052865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48043 (local address 127.1.132.190:36821)
0923 22:16:41.053066 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.053075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.053110 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:16:41.053240 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.053247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.053305 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.054174 (+ 869us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:41.054286 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.055615 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.056620 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.057022 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.057119 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.059274 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:41.059279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:41.059279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:41.059291 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:41.060855 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:41.061222 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:41.061225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:41.061226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:41.061319 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:41.061515 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:41.061515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:41.061516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:41.061753 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:41.061904 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.062017 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.062209 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260923 22:16:41.063220 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:41.069880 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:41.064485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50681 (local address 127.1.132.190:36821)
0923 22:16:41.064528 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:41.064530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:41.064543 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:16:41.064611 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:41.064612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:41.064634 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:41.064679 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:41.065126 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.065603 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.066256 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:41.066471 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:41.066524 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:41.067597 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:41.067599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:41.067599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:41.067606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:41.068864 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:41.069183 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:41.069184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:41.069184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:41.069238 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:41.069419 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:41.069420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:41.069420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:41.069524 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:41.069629 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:41.069732 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 22:16:41.069789 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:16:41.083673 23195 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:41.084617 23195 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:41.093804 23195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 859cbac7e986458ba5e655aab54811e0: Generated new TSK 0
I20260923 22:16:41.094139 23195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:41.102412 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23113
W20260923 22:16:41.106530 23098 connection.cc:535] client connection to 127.1.132.190:36821 recv error: Network error: failed to read from TLS socket (remote: 127.1.132.190:36821): Connection reset by peer (error 104)
2026-09-23T22:16:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:41 dist-test-slave-l4q5 krb5kdc[23212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:41 dist-test-slave-l4q5 krb5kdc[23212](info): set up 2 sockets
Sep 23 22:16:41 dist-test-slave-l4q5 krb5kdc[23212](info): commencing operation
krb5kdc: starting...
W20260923 22:16:42.076104 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:42 dist-test-slave-l4q5 krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:42.198882 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:37795
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:42.278787 23225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:42.279027 23225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:42.279063 23225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:42.280359 23225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:42.280412 23225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:42.280431 23225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:42.280447 23225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:42.280462 23225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:42.282212 23225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:37795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:42.282501 23225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:42.282722 23225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:42.287490 23230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:42.287509 23231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:42.287493 23233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:42.287720 23225 server_base.cc:1034] running on GCE node
I20260923 22:16:42.287957 23225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:42.288292 23225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:42.289499 23225 hybrid_clock.cc:648] HybridClock initialized: now 1790201802289482 us; error 33 us; skew 500 ppm
Sep 23 22:16:42 dist-test-slave-l4q5 krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201802, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:42.296993 23225 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:42.297494 23225 webserver.cc:460] Webserver started at http://127.1.132.190:37825/ using document root <none> and password file <none>
I20260923 22:16:42.297709 23225 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:42.297760 23225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:42.297885 23225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:42.298952 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "42686d7552c64563bca3dd568ac9e2af"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-l4q5"
server_key: "6a4f531251cf8bb162f590dfff4d32c4"
server_key_iv: "52bb86208f332f0e367928f1fc448ea9"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.299332 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "42686d7552c64563bca3dd568ac9e2af"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-l4q5"
server_key: "6a4f531251cf8bb162f590dfff4d32c4"
server_key_iv: "52bb86208f332f0e367928f1fc448ea9"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.300841 23225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:16:42.301716 23240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:42.301925 23225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:42.302026 23225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "42686d7552c64563bca3dd568ac9e2af"
format_stamp: "Formatted at 2026-09-23 22:16:42 on dist-test-slave-l4q5"
server_key: "6a4f531251cf8bb162f590dfff4d32c4"
server_key_iv: "52bb86208f332f0e367928f1fc448ea9"
server_key_version: "encryptionkey@0"
I20260923 22:16:42.302098 23225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:42.316747 23225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:42.318775 23225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:42.318920 23225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:42.322836 23225 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:37795
I20260923 22:16:42.322919 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:37795 every 8 connection(s)
I20260923 22:16:42.323223 23225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:42.323781 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23225
I20260923 22:16:42.323885 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:42.324079 1554 external_mini_cluster.cc:1428] Setting key 406579387be5a19b48dfbaf5d56718ee
I20260923 22:16:42.324283 23293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:42.327447 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af: Bootstrap starting.
I20260923 22:16:42.328152 23293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:42.328495 23293 log.cc:824] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:42.329267 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af: No bootstrap required, opened a new log
I20260923 22:16:42.329828 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.324618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44785 (local address 127.1.132.190:37795)
0923 22:16:42.324804 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:42.324813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:42.324838 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:16:42.325317 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:42.325324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:42.325371 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:42.325481 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:42.325730 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.326590 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.327641 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.328333 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.328458 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:42.329151 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:42.329155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:42.329155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:42.329354 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:42.329396 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:42.329412 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:42.329491 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:42.329577 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:16:42.329604 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260923 22:16:42.330973 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:42.331641 23293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } }
I20260923 22:16:42.331826 23293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:42.331861 23293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42686d7552c64563bca3dd568ac9e2af, State: Initialized, Role: FOLLOWER
I20260923 22:16:42.331984 23293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } }
I20260923 22:16:42.332067 23293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:42.332105 23293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:42.332159 23293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:42.332933 23293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } }
I20260923 22:16:42.333027 23293 leader_election.cc:302] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42686d7552c64563bca3dd568ac9e2af; no voters:
I20260923 22:16:42.333185 23293 leader_election.cc:288] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:42.333212 23303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:42.333362 23303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [term 1 LEADER]: Becoming Leader. State: Replica: 42686d7552c64563bca3dd568ac9e2af, State: Running, Role: LEADER
I20260923 22:16:42.333446 23293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:42.333504 23303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } }
I20260923 22:16:42.333941 23303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42686d7552c64563bca3dd568ac9e2af. Latest consensus state: current_term: 1 leader_uuid: "42686d7552c64563bca3dd568ac9e2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } } }
I20260923 22:16:42.333952 23305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42686d7552c64563bca3dd568ac9e2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42686d7552c64563bca3dd568ac9e2af" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 37795 } } }
I20260923 22:16:42.334105 23303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:42.334115 23305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:42.334372 23312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:42.334934 23312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:42.335827 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:42.332478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43283 (local address 127.1.132.190:37795)
0923 22:16:42.332517 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:42.332518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:42.332624 (+ 106us) server_negotiation.cc:407] Connection header received
0923 22:16:42.332681 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:42.332682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:42.332690 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:42.332733 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:42.332867 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.333549 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.334507 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:42.334808 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:42.334872 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:42.335602 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:42.335603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:42.335603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:42.335608 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:42.335622 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:42.335627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:42.335677 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:42.335747 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:16:42.335756 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:42.337148 23312 catalog_manager.cc:1347] Generated new cluster ID: a20dfdc9a2444161bdedec9257fb35df
I20260923 22:16:42.337198 23312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:42.351177 23312 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:42.351954 23312 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:42.355646 23312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42686d7552c64563bca3dd568ac9e2af: Generated new TSK 0
I20260923 22:16:42.355832 23312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:42.358093 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23225
W20260923 22:16:42.361835 23210 connection.cc:535] client connection to 127.1.132.190:37795 recv error: Network error: failed to read from TLS socket (remote: 127.1.132.190:37795): Connection reset by peer (error 104)
2026-09-23T22:16:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:42 dist-test-slave-l4q5 krb5kdc[23324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:42 dist-test-slave-l4q5 krb5kdc[23324](info): set up 2 sockets
Sep 23 22:16:42 dist-test-slave-l4q5 krb5kdc[23324](info): commencing operation
krb5kdc: starting...
W20260923 22:16:43.328075 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:43 dist-test-slave-l4q5 krb5kdc[23324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:43.447121 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45733
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:43.529556 23337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:43.529774 23337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:43.529798 23337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:43.530969 23337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:43.531015 23337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:43.531029 23337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:43.531040 23337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:43.531051 23337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:43.532589 23337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:45733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:43.532831 23337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:43.533049 23337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:43.537704 23342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.537699 23345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:43.537899 23343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:43.537823 23337 server_base.cc:1034] running on GCE node
I20260923 22:16:43.538178 23337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:43.538452 23337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:43.539595 23337 hybrid_clock.cc:648] HybridClock initialized: now 1790201803539573 us; error 35 us; skew 500 ppm
Sep 23 22:16:43 dist-test-slave-l4q5 krb5kdc[23324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201803, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:43.542062 23337 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:43.542451 23337 webserver.cc:460] Webserver started at http://127.1.132.190:43969/ using document root <none> and password file <none>
I20260923 22:16:43.542640 23337 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:43.542681 23337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:43.542773 23337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:43.543807 23337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "e71a67d89fa5464baf28e734ded31fa1"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-l4q5"
server_key: "f5c97f90e225f54ab91b09c569c493f2"
server_key_iv: "29c19d8a8f7385c83307758d2d1b1237"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.544099 23337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "e71a67d89fa5464baf28e734ded31fa1"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-l4q5"
server_key: "f5c97f90e225f54ab91b09c569c493f2"
server_key_iv: "29c19d8a8f7385c83307758d2d1b1237"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.545358 23337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:43.546047 23352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:43.546221 23337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:43.546298 23337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "e71a67d89fa5464baf28e734ded31fa1"
format_stamp: "Formatted at 2026-09-23 22:16:43 on dist-test-slave-l4q5"
server_key: "f5c97f90e225f54ab91b09c569c493f2"
server_key_iv: "29c19d8a8f7385c83307758d2d1b1237"
server_key_version: "encryptionkey@0"
I20260923 22:16:43.546371 23337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:43.549576 23337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:43.551862 23337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:43.551980 23337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:43.555590 23337 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:45733
I20260923 22:16:43.555640 23404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:45733 every 8 connection(s)
I20260923 22:16:43.555934 23337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:43.556982 23405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:43.559429 23405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1: Bootstrap starting.
I20260923 22:16:43.560060 23405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:43.560360 23405 log.cc:824] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:43.561062 23405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1: No bootstrap required, opened a new log
I20260923 22:16:43.561975 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23337
I20260923 22:16:43.562086 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:43.562251 1554 external_mini_cluster.cc:1428] Setting key dfe355bac80fdf60933123ef43eeb9d8
I20260923 22:16:43.562932 23405 raft_consensus.cc:348] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } }
I20260923 22:16:43.563145 23405 raft_consensus.cc:374] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:43.563181 23405 raft_consensus.cc:729] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71a67d89fa5464baf28e734ded31fa1, State: Initialized, Role: FOLLOWER
I20260923 22:16:43.563316 23405 consensus_queue.cc:260] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } }
I20260923 22:16:43.563397 23405 raft_consensus.cc:388] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:43.563427 23405 raft_consensus.cc:482] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:43.563510 23405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:43.564498 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.562801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52791 (local address 127.1.132.190:45733)
0923 22:16:43.563015 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.563024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.563061 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:16:43.563174 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.563181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.563212 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.563351 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.564044 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.564048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.564049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:43.564054 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.564099 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:43.564110 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.564171 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.564265 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.564308 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260923 22:16:43.564618 23405 raft_consensus.cc:504] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } }
I20260923 22:16:43.564728 23405 leader_election.cc:302] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71a67d89fa5464baf28e734ded31fa1; no voters:
I20260923 22:16:43.564890 23405 leader_election.cc:288] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:43.564929 23410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:43.565169 23405 sys_catalog.cc:564] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:43.565199 23410 raft_consensus.cc:686] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [term 1 LEADER]: Becoming Leader. State: Replica: e71a67d89fa5464baf28e734ded31fa1, State: Running, Role: LEADER
I20260923 22:16:43.565277 23410 consensus_queue.cc:237] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } }
I20260923 22:16:43.565598 23410 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e71a67d89fa5464baf28e734ded31fa1. Latest consensus state: current_term: 1 leader_uuid: "e71a67d89fa5464baf28e734ded31fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } } }
I20260923 22:16:43.565673 23410 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:43.565650 23411 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e71a67d89fa5464baf28e734ded31fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71a67d89fa5464baf28e734ded31fa1" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 45733 } } }
I20260923 22:16:43.566010 23411 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:43.566087 23419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:43.566488 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:43.566854 23419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:43.573686 23419 catalog_manager.cc:1347] Generated new cluster ID: bc5a1a467da54af6bc5b5b1b4654a3f7
I20260923 22:16:43.573766 23419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:43.574055 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:43.573075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40249 (local address 127.1.132.190:45733)
0923 22:16:43.573111 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:43.573113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:43.573122 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:43.573146 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:43.573147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:43.573153 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:43.573215 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:43.573833 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:43.573835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:43.573835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:43.573839 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:43.573855 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:43.573860 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:43.573887 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:43.573966 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:16:43.573980 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:43.585259 23419 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:43.585435 23419 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:43.598843 23419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e71a67d89fa5464baf28e734ded31fa1: Generated new TSK 0
I20260923 22:16:43.599078 23419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:43.607767 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23337
2026-09-23T22:16:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:43 dist-test-slave-l4q5 krb5kdc[23436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:43 dist-test-slave-l4q5 krb5kdc[23436](info): set up 2 sockets
Sep 23 22:16:43 dist-test-slave-l4q5 krb5kdc[23436](info): commencing operation
krb5kdc: starting...
W20260923 22:16:44.564422 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:44.683316 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33823
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:33823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:44.763785 23449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:44.764040 23449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:44.764077 23449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:44.765369 23449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:44.765429 23449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:44.765447 23449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:44.765462 23449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:44.765479 23449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:44.767119 23449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:33823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:44.767403 23449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:44.767629 23449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:44.773437 23457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:44.773707 23455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:44.773797 23449 server_base.cc:1034] running on GCE node
W20260923 22:16:44.773437 23454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:44.774209 23449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:44.774668 23449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:44.775840 23449 hybrid_clock.cc:648] HybridClock initialized: now 1790201804775821 us; error 38 us; skew 500 ppm
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201804, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:44.778676 23449 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:44.779139 23449 webserver.cc:460] Webserver started at http://127.1.132.190:41687/ using document root <none> and password file <none>
I20260923 22:16:44.779348 23449 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:44.779397 23449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:44.779522 23449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:44.780650 23449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db"
format_stamp: "Formatted at 2026-09-23 22:16:44 on dist-test-slave-l4q5"
server_key: "8c5ca5b2bd820ad81da1ab691e50bc9d"
server_key_iv: "f0cb72ba44a8b63bf6ed3a2a0db93b1e"
server_key_version: "encryptionkey@0"
I20260923 22:16:44.780977 23449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db"
format_stamp: "Formatted at 2026-09-23 22:16:44 on dist-test-slave-l4q5"
server_key: "8c5ca5b2bd820ad81da1ab691e50bc9d"
server_key_iv: "f0cb72ba44a8b63bf6ed3a2a0db93b1e"
server_key_version: "encryptionkey@0"
I20260923 22:16:44.782336 23449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:44.783248 23464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:44.783492 23449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:44.783586 23449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db"
format_stamp: "Formatted at 2026-09-23 22:16:44 on dist-test-slave-l4q5"
server_key: "8c5ca5b2bd820ad81da1ab691e50bc9d"
server_key_iv: "f0cb72ba44a8b63bf6ed3a2a0db93b1e"
server_key_version: "encryptionkey@0"
I20260923 22:16:44.783669 23449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:44.793294 23449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:44.795446 23449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:44.795585 23449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:44.799485 23449 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:33823
I20260923 22:16:44.799530 23516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:33823 every 8 connection(s)
I20260923 22:16:44.799851 23449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:44.800947 23517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:44.803164 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db: Bootstrap starting.
I20260923 22:16:44.803699 23517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:44.803920 23517 log.cc:824] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:44.804629 23517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db: No bootstrap required, opened a new log
I20260923 22:16:44.806398 23517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } }
I20260923 22:16:44.806587 23517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:44.806612 23517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b2ddd15b3e4f618ea0ca5c7332a0db, State: Initialized, Role: FOLLOWER
I20260923 22:16:44.806733 23517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } }
I20260923 22:16:44.806807 23517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:44.806845 23517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:44.806881 23517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:44.807762 23517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } }
I20260923 22:16:44.807855 23517 leader_election.cc:302] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65b2ddd15b3e4f618ea0ca5c7332a0db; no voters:
I20260923 22:16:44.807989 23517 leader_election.cc:288] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:44.808053 23520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:44.808261 23517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:44.808290 23520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [term 1 LEADER]: Becoming Leader. State: Replica: 65b2ddd15b3e4f618ea0ca5c7332a0db, State: Running, Role: LEADER
I20260923 22:16:44.808298 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23449
I20260923 22:16:44.808478 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:44.808463 23520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } }
I20260923 22:16:44.808723 1554 external_mini_cluster.cc:1428] Setting key a6768f9897a820f2378b8143347a96b7
I20260923 22:16:44.808866 23522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65b2ddd15b3e4f618ea0ca5c7332a0db. Latest consensus state: current_term: 1 leader_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } } }
I20260923 22:16:44.808974 23522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:44.809232 23529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:44.809316 23521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2ddd15b3e4f618ea0ca5c7332a0db" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33823 } } }
I20260923 22:16:44.809404 23521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:44.809729 23529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:44.811246 23529 catalog_manager.cc:1347] Generated new cluster ID: f3075ee24f334a0ca71c625555f1f5db
I20260923 22:16:44.811298 23529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:44.815795 23529 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:44.816344 23529 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:44.822659 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.809371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39713 (local address 127.1.132.190:33823)
0923 22:16:44.809563 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.809574 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.812452 (+ 2878us) server_negotiation.cc:407] Connection header received
0923 22:16:44.812606 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.812613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.812673 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.812834 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:44.813506 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.814962 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.815804 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.816507 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.816622 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.819333 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:44.819337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:44.819337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:44.819350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:44.821135 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:44.821531 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:44.821533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:44.821534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:44.821623 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:44.821797 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:44.821798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:44.821798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:44.822045 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:44.822176 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.822278 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.822424 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260923 22:16:44.823668 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:44.824231 23529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65b2ddd15b3e4f618ea0ca5c7332a0db: Generated new TSK 0
I20260923 22:16:44.824389 23529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:44.831326 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:44.825133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44707 (local address 127.1.132.190:33823)
0923 22:16:44.825184 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:44.825186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:44.825250 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:16:44.825321 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:44.825321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:44.825330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:44.825373 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:44.826028 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.826546 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.827566 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:44.827761 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:44.827806 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:44.829187 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:44.829188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:44.829189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:44.829194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:44.830372 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:44.830733 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:44.830734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:44.830734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:44.830774 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:44.830934 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:44.830934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:44.830934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:44.831020 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:44.831093 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:44.831206 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 22:16:44.831264 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:16:44.833384 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23449
2026-09-23T22:16:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1225 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23548](info): set up 2 sockets
Sep 23 22:16:44 dist-test-slave-l4q5 krb5kdc[23548](info): commencing operation
krb5kdc: starting...
W20260923 22:16:45.816234 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:45 dist-test-slave-l4q5 krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:45.945830 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40261
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:33131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:40261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:46.033242 23561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:46.033470 23561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:46.033495 23561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:46.034770 23561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:46.034825 23561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:46.034838 23561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:46.034850 23561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:46.034861 23561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:46.036541 23561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:33131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:40261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:40261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:46.036808 23561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:46.037048 23561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:46.042372 23569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:46.042408 23561 server_base.cc:1034] running on GCE node
W20260923 22:16:46.042392 23566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:46.042372 23567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:46.042918 23561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:46.043236 23561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:46.044423 23561 hybrid_clock.cc:648] HybridClock initialized: now 1790201806044404 us; error 31 us; skew 500 ppm
Sep 23 22:16:46 dist-test-slave-l4q5 krb5kdc[23548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:46.047129 23561 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:46.047622 23561 webserver.cc:460] Webserver started at http://127.1.132.190:35543/ using document root <none> and password file <none>
I20260923 22:16:46.047830 23561 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:46.047878 23561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:46.048012 23561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:46.049151 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "c000a7b319fa4930b5db5966e2443801"
format_stamp: "Formatted at 2026-09-23 22:16:46 on dist-test-slave-l4q5"
server_key: "83cf8dafeacf3f1ee9a0b2e6c61484f8"
server_key_iv: "719cd5c66f67153792e9ae126a86a64c"
server_key_version: "encryptionkey@0"
I20260923 22:16:46.049494 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "c000a7b319fa4930b5db5966e2443801"
format_stamp: "Formatted at 2026-09-23 22:16:46 on dist-test-slave-l4q5"
server_key: "83cf8dafeacf3f1ee9a0b2e6c61484f8"
server_key_iv: "719cd5c66f67153792e9ae126a86a64c"
server_key_version: "encryptionkey@0"
I20260923 22:16:46.050824 23561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:46.051604 23576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:46.051756 23561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:46.051842 23561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "c000a7b319fa4930b5db5966e2443801"
format_stamp: "Formatted at 2026-09-23 22:16:46 on dist-test-slave-l4q5"
server_key: "83cf8dafeacf3f1ee9a0b2e6c61484f8"
server_key_iv: "719cd5c66f67153792e9ae126a86a64c"
server_key_version: "encryptionkey@0"
I20260923 22:16:46.051916 23561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:46.075055 23561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:46.077595 23561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:46.077768 23561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:46.081723 23561 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:40261
I20260923 22:16:46.081761 23628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:40261 every 8 connection(s)
I20260923 22:16:46.082122 23561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:46.082981 23629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:46.085363 23629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801: Bootstrap starting.
I20260923 22:16:46.085951 23629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:46.086220 23629 log.cc:824] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:46.086927 23629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801: No bootstrap required, opened a new log
I20260923 22:16:46.088631 23629 raft_consensus.cc:348] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } }
I20260923 22:16:46.088784 23629 raft_consensus.cc:374] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:46.088806 23629 raft_consensus.cc:729] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c000a7b319fa4930b5db5966e2443801, State: Initialized, Role: FOLLOWER
I20260923 22:16:46.088940 23629 consensus_queue.cc:260] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } }
I20260923 22:16:46.089015 23629 raft_consensus.cc:388] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:46.089036 23629 raft_consensus.cc:482] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:46.089063 23629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:46.089907 23629 raft_consensus.cc:504] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } }
I20260923 22:16:46.089995 23629 leader_election.cc:302] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c000a7b319fa4930b5db5966e2443801; no voters:
I20260923 22:16:46.090163 23629 leader_election.cc:288] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:46.090252 23632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:46.090402 23629 sys_catalog.cc:564] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:46.090425 23632 raft_consensus.cc:686] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [term 1 LEADER]: Becoming Leader. State: Replica: c000a7b319fa4930b5db5966e2443801, State: Running, Role: LEADER
I20260923 22:16:46.090583 23632 consensus_queue.cc:237] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } }
I20260923 22:16:46.090997 23634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c000a7b319fa4930b5db5966e2443801. Latest consensus state: current_term: 1 leader_uuid: "c000a7b319fa4930b5db5966e2443801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } } }
I20260923 22:16:46.091109 23634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:46.090997 23633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c000a7b319fa4930b5db5966e2443801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000a7b319fa4930b5db5966e2443801" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 40261 } } }
I20260923 22:16:46.091183 23633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801 [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:46.091382 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23561
I20260923 22:16:46.091490 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:46.091324 23641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:46.091671 1554 external_mini_cluster.cc:1428] Setting key a9e5a785c0e51534c38a98ccec3eaed2
I20260923 22:16:46.092028 23641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:46.093650 23641 catalog_manager.cc:1347] Generated new cluster ID: 57806d59278a49abae0b301ea57b0d83
I20260923 22:16:46.093706 23641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:46.097023 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:46.092314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59487 (local address 127.1.132.190:40261)
0923 22:16:46.092749 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:46.092758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:46.092786 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:16:46.092903 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:46.092909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:46.092966 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:46.093109 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:46.093316 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:46.094366 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:46.095138 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:46.095598 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:46.095709 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:46.096378 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:46.096382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:46.096382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:46.096567 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:46.096599 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:46.096611 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:46.096665 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:46.096744 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:16:46.096809 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":61,"threads_started":1}
I20260923 22:16:46.097877 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:46.101788 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:46.099229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45067 (local address 127.1.132.190:40261)
0923 22:16:46.099263 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:46.099265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:46.099272 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:46.099308 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:46.099309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:46.099319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:46.099362 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:46.099488 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:46.099955 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:46.100694 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:46.100906 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:46.100961 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:46.101560 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:46.101561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:46.101562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:46.101566 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:46.101581 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:46.101585 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:46.101629 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:46.101685 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:16:46.101710 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:46.105666 23641 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:46.106212 23641 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:46.116523 23641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c000a7b319fa4930b5db5966e2443801: Generated new TSK 0
I20260923 22:16:46.116752 23641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:46.124843 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23561
2026-09-23T22:16:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:46 dist-test-slave-l4q5 krb5kdc[23660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:46 dist-test-slave-l4q5 krb5kdc[23660](info): set up 2 sockets
Sep 23 22:16:46 dist-test-slave-l4q5 krb5kdc[23660](info): commencing operation
krb5kdc: starting...
W20260923 22:16:47.096123 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:47 dist-test-slave-l4q5 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:47.224026 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36575
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:42617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:36575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:47.317556 23673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:47.317804 23673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:47.317829 23673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:47.319077 23673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:47.319128 23673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:47.319140 23673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:47.319151 23673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:47.319162 23673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:47.320801 23673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:42617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:36575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:36575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:47.321049 23673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:47.321244 23673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:47.326483 23678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:47.326575 23681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:47.326581 23679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:47.326719 23673 server_base.cc:1034] running on GCE node
I20260923 22:16:47.327172 23673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:47.327472 23673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:47.328616 23673 hybrid_clock.cc:648] HybridClock initialized: now 1790201807328603 us; error 32 us; skew 500 ppm
Sep 23 22:16:47 dist-test-slave-l4q5 krb5kdc[23660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:47.336328 23673 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:47.336880 23673 webserver.cc:460] Webserver started at http://127.1.132.190:43797/ using document root <none> and password file <none>
I20260923 22:16:47.337092 23673 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:47.337143 23673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:47.337373 23673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:47.338487 23673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "e54f86061df04d97b7835e949a6a874d"
format_stamp: "Formatted at 2026-09-23 22:16:47 on dist-test-slave-l4q5"
server_key: "ab4afcf1a29f6a31afad2cde8983a0c9"
server_key_iv: "400708c605a353aca6698624512672c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:47.338831 23673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "e54f86061df04d97b7835e949a6a874d"
format_stamp: "Formatted at 2026-09-23 22:16:47 on dist-test-slave-l4q5"
server_key: "ab4afcf1a29f6a31afad2cde8983a0c9"
server_key_iv: "400708c605a353aca6698624512672c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:47.340451 23673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:16:47.341388 23688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:47.341740 23673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:47.341850 23673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "e54f86061df04d97b7835e949a6a874d"
format_stamp: "Formatted at 2026-09-23 22:16:47 on dist-test-slave-l4q5"
server_key: "ab4afcf1a29f6a31afad2cde8983a0c9"
server_key_iv: "400708c605a353aca6698624512672c8"
server_key_version: "encryptionkey@0"
I20260923 22:16:47.341943 23673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:47.345621 23673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:47.347826 23673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:47.347947 23673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:47.351954 23673 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:36575
I20260923 22:16:47.352048 23740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:36575 every 8 connection(s)
I20260923 22:16:47.352427 23673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:47.353595 23741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:47.356037 23741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d: Bootstrap starting.
I20260923 22:16:47.356678 23741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:47.356993 23741 log.cc:824] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:47.358179 23741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d: No bootstrap required, opened a new log
I20260923 22:16:47.359658 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23673
I20260923 22:16:47.359771 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:47.359956 1554 external_mini_cluster.cc:1428] Setting key 8160d6db88b5401b858706f4a3a98ae3
I20260923 22:16:47.360109 23741 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } }
I20260923 22:16:47.360322 23741 raft_consensus.cc:374] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:16:47.360350 23741 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54f86061df04d97b7835e949a6a874d, State: Initialized, Role: FOLLOWER
I20260923 22:16:47.360483 23741 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } }
I20260923 22:16:47.360564 23741 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:16:47.360599 23741 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:16:47.360646 23741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:47.361944 23741 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } }
I20260923 22:16:47.362053 23741 leader_election.cc:302] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54f86061df04d97b7835e949a6a874d; no voters:
I20260923 22:16:47.362238 23741 leader_election.cc:288] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:16:47.362283 23746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:47.362429 23746 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [term 1 LEADER]: Becoming Leader. State: Replica: e54f86061df04d97b7835e949a6a874d, State: Running, Role: LEADER
I20260923 22:16:47.362525 23741 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:47.362680 23746 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } }
I20260923 22:16:47.362849 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:47.360568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49483 (local address 127.1.132.190:36575)
0923 22:16:47.360792 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:47.360800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:47.360828 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:16:47.361108 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:47.361117 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:47.361172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:47.361321 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:47.362190 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:47.362194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:47.362194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:47.362199 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:47.362249 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:47.362265 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:47.362341 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:47.362451 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:16:47.362554 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260923 22:16:47.363090 23747 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e54f86061df04d97b7835e949a6a874d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } } }
I20260923 22:16:47.363169 23747 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:47.363317 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54f86061df04d97b7835e949a6a874d. Latest consensus state: current_term: 1 leader_uuid: "e54f86061df04d97b7835e949a6a874d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f86061df04d97b7835e949a6a874d" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 36575 } } }
I20260923 22:16:47.363377 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:47.363483 23755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:47.363799 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:47.364004 23755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:47.365686 23755 catalog_manager.cc:1347] Generated new cluster ID: 830558ec4ecf474ab101bb889e9a1c7d
I20260923 22:16:47.365731 23755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:47.366832 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:47.365325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39449 (local address 127.1.132.190:36575)
0923 22:16:47.365380 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:47.365382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:47.365807 (+ 425us) server_negotiation.cc:407] Connection header received
0923 22:16:47.365902 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:47.365903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:47.365909 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:47.365937 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:47.366603 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:47.366605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:47.366605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:16:47.366609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:47.366624 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:16:47.366629 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:47.366658 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:47.366759 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:16:47.366774 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:16:47.384105 23755 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:47.384325 23755 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:47.390215 23755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e54f86061df04d97b7835e949a6a874d: Generated new TSK 0
I20260923 22:16:47.390434 23755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:16:47.398821 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23673
2026-09-23T22:16:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1273 ms)
[----------] 18 tests from AuthTokenIssuingTest (22444 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:47 dist-test-slave-l4q5 krb5kdc[23772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:47 dist-test-slave-l4q5 krb5kdc[23772](info): set up 2 sockets
Sep 23 22:16:47 dist-test-slave-l4q5 krb5kdc[23772](info): commencing operation
krb5kdc: starting...
W20260923 22:16:48.360980 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:48.487998 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33599
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:48.569715 23785 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:48.570005 23785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:48.570035 23785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:48.570063 23785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:48.571365 23785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:48.571415 23785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:48.571434 23785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:48.571450 23785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:48.571465 23785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:48.573114 23785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:33599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:48.573387 23785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:48.573613 23785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:48.578693 23791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:48.578856 23785 server_base.cc:1034] running on GCE node
W20260923 22:16:48.578693 23793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:48.578809 23790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:48.579226 23785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:48.579541 23785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:48.580699 23785 hybrid_clock.cc:648] HybridClock initialized: now 1790201808580675 us; error 42 us; skew 500 ppm
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:48.583132 23785 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:48.583559 23785 webserver.cc:460] Webserver started at http://127.1.132.190:38093/ using document root <none> and password file <none>
I20260923 22:16:48.583760 23785 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:48.583807 23785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:48.583940 23785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:48.585037 23785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "ff202e85c18448588072837f236d2bcd"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "e96941f19d805a168595d651c46b3ea0"
server_key_iv: "74be41ffd49038f70dc9ab56011f76f4"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.585376 23785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "ff202e85c18448588072837f236d2bcd"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "e96941f19d805a168595d651c46b3ea0"
server_key_iv: "74be41ffd49038f70dc9ab56011f76f4"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.586689 23785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:16:48.587509 23800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:48.587742 23785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:16:48.587836 23785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "ff202e85c18448588072837f236d2bcd"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "e96941f19d805a168595d651c46b3ea0"
server_key_iv: "74be41ffd49038f70dc9ab56011f76f4"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.587942 23785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:48.599799 23785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:48.601997 23785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:48.602197 23785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:48.606253 23785 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:33599
I20260923 22:16:48.606299 23852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:33599 every 8 connection(s)
I20260923 22:16:48.606637 23785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:48.607671 23853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:48.609463 23853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.612954 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23785
I20260923 22:16:48.613065 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:48.613226 1554 external_mini_cluster.cc:1428] Setting key c3436bdbb7aa703cafbffc7bee41148a
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:48.620121 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.610138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33599 (local address 127.0.0.1:38116)
0923 22:16:48.610348 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.610397 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.610901 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.611290 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.611293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.611569 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.611728 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.611732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.612795 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.612796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.613564 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.613567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.614045 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.614046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.614109 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.614618 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.614627 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.616261 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.618488 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.618495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.618497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.618701 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.618992 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.618993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.618993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.619087 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.619591 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.619598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.619666 (+ 68us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.619760 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.619944 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260923 22:16:48.620141 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.610229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.1.132.190:33599)
0923 22:16:48.610384 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.610394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.610506 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:16:48.611083 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.611085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.611106 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.611249 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.611831 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.612701 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.613641 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.613944 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.614044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.616365 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.616376 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.616377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.616385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.618392 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.618784 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.618787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.618788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.618899 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.619161 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.619162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.619162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.619502 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.619591 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.619819 (+ 228us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.620039 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.1.132.189@KRBTEST.COM; defaulting to no policy
I20260923 22:16:48.620944 23853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.621321 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.621185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54668)
0923 22:16:48.621252 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.621299 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 22:16:48.621505 23801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
I20260923 22:16:48.621994 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.621879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54670)
0923 22:16:48.621912 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.621940 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 22:16:48.622094 23853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:41167: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
I20260923 22:16:48.638557 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:41167
--webserver_interface=127.1.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:48.671341 23853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } attempt: 1
I20260923 22:16:48.671989 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.671772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54676)
0923 22:16:48.671909 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.671956 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260923 22:16:48.672364 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.672278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54686)
0923 22:16:48.672315 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.672341 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:16:48.672464 23853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:41167: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
W20260923 22:16:48.720077 23860 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:48.720345 23860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:48.720361 23860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:48.720383 23860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:48.721567 23860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:48.721607 23860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:48.721621 23860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:48.721630 23860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:48.721642 23860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:48.723182 23860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:41167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:48.723409 23860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:48.723598 23860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:48.728407 23865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:48.728410 23868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:48.728410 23866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:48.728529 23860 server_base.cc:1034] running on GCE node
I20260923 22:16:48.728919 23860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:48.729256 23860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:48.730407 23860 hybrid_clock.cc:648] HybridClock initialized: now 1790201808730392 us; error 31 us; skew 500 ppm
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:48.733141 23860 init.cc:377] Logged in from keytab as kudu/127.1.132.189@KRBTEST.COM (short username kudu)
I20260923 22:16:48.733611 23860 webserver.cc:460] Webserver started at http://127.1.132.189:34917/ using document root <none> and password file <none>
I20260923 22:16:48.733816 23860 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:48.733855 23860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:48.733961 23860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:48.734980 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data/instance:
uuid: "91b547a5954e459fb7fe421693faff2e"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "6b0c7dd81310945f8473b078708b0964"
server_key_iv: "3b1031cf6909511237f8f0c9f7c5903c"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.735261 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal/instance:
uuid: "91b547a5954e459fb7fe421693faff2e"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "6b0c7dd81310945f8473b078708b0964"
server_key_iv: "3b1031cf6909511237f8f0c9f7c5903c"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.736558 23860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:16:48.737277 23875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:48.737449 23860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:16:48.737521 23860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "91b547a5954e459fb7fe421693faff2e"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "6b0c7dd81310945f8473b078708b0964"
server_key_iv: "3b1031cf6909511237f8f0c9f7c5903c"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.737576 23860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:48.740625 23853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } attempt: 2
I20260923 22:16:48.741070 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.740933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54700)
0923 22:16:48.741000 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.741040 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:16:48.741433 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.741316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54712)
0923 22:16:48.741380 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.741411 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 22:16:48.741521 23853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:41167: Network error: Client connection negotiation failed: client connection to 127.1.132.189:41167: connect: Connection refused (error 111)
I20260923 22:16:48.756291 23860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:48.758618 23860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:48.758771 23860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:48.762748 23860 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.189:41167
I20260923 22:16:48.762780 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.189:41167 every 8 connection(s)
I20260923 22:16:48.763147 23860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
I20260923 22:16:48.763484 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23860
I20260923 22:16:48.763590 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-1/wal/instance
I20260923 22:16:48.763756 1554 external_mini_cluster.cc:1428] Setting key 412657f2393abe75ae599a525aa1234e
I20260923 22:16:48.764343 23928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:48.765974 23928 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.188@KRBTEST.COM" created.
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:48.777674 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.766743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.1.132.190:33599)
0923 22:16:48.766805 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.766807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.767145 (+ 338us) server_negotiation.cc:407] Connection header received
0923 22:16:48.767686 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.767687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.767703 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.767759 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.768509 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.768991 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.770062 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.770265 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.770323 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.774768 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.774771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.774771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.774782 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.776239 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.776698 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.776700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.776700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.776766 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.777007 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.777008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.777008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.777149 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.777209 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.777489 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.777580 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:16:48.777858 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.766705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33599 (local address 127.0.0.1:38128)
0923 22:16:48.766988 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.767033 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.767570 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.767849 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.767853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.768127 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.768408 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.768413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.769083 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.769084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.769964 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.769967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.770361 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.770362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.770492 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.771124 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.771134 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.774639 (+ 3505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.776336 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.776346 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.776348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.776617 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.776827 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.776827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.776828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.776941 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.777216 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.777224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.777366 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.777458 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.777642 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":151,"threads_started":1}
I20260923 22:16:48.778391 23928 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:48.787401 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.778640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54722)
0923 22:16:48.778702 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.778715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.778768 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.778965 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.778966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.779115 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.779182 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.779184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.780045 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.780046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.780885 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.780888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.781418 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.781419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.781493 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.781973 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.781978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.784399 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.786296 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.786297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.786297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.786468 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.786683 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.786683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.786683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.786730 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.787113 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.787114 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.787165 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.787237 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.787327 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:16:48.787555 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.778678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54722 (local address 127.1.132.189:41167)
0923 22:16:48.778825 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.778830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.778849 (+ 19us) server_negotiation.cc:407] Connection header received
0923 22:16:48.778869 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.778871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.778881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.778942 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.779254 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.779953 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.780993 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.781312 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.781370 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.784516 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.784528 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.784529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.784538 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.786203 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.786564 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.786568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.786568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.786624 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.786773 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.786774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.786774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.787027 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.787096 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.787310 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.787384 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260923 22:16:48.788266 23928 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.788635 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.788517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40930)
0923 22:16:48.788571 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.788615 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:16:48.788757 23878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
I20260923 22:16:48.789139 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.789060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40942)
0923 22:16:48.789098 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.789122 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 22:16:48.789237 23928 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:33741: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
I20260923 22:16:48.791929 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:33741
--webserver_interface=127.1.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:46365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:48.832700 23853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } attempt: 3
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:48.838557 23928 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } attempt: 1
I20260923 22:16:48.839026 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.838882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40952)
0923 22:16:48.838951 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.838990 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 22:16:48.839392 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.839289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40968)
0923 22:16:48.839335 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.839364 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:16:48.839515 23928 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:33741: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
I20260923 22:16:48.840483 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.833127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54736)
0923 22:16:48.833184 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.833197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.833242 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.833398 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.833399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.833562 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.833633 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.833635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.834311 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.834313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.834991 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.834995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.835449 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.835450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.835520 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.835979 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.835984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.837442 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.839202 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.839203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.839204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.839430 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.839690 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.839692 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.839692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.839769 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.840117 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.840118 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.840210 (+ 92us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.840278 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.840386 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:16:48.840487 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.833217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.1.132.189:41167)
0923 22:16:48.833287 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.833289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.833298 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:48.833324 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.833325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.833336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.833380 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.833720 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.834225 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.835113 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.835351 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.835416 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.837545 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.837547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.837548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.837556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.839064 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.839514 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.839515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.839516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.839603 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.839862 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.839863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.839864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.840006 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.840099 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.840303 (+ 204us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.840411 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:16:48.840826 23853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.841130 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.841038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40984)
0923 22:16:48.841082 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.841107 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:16:48.841418 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.841350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:40990)
0923 22:16:48.841382 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.841404 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:16:48.841506 23853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:33741: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
I20260923 22:16:48.872665 23853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } attempt: 1
I20260923 22:16:48.873214 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.873061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41004)
0923 22:16:48.873128 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.873177 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 22:16:48.873656 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.873536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41018)
0923 22:16:48.873601 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.873632 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 22:16:48.873767 23853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:33741: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
W20260923 22:16:48.876609 23935 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:48.876832 23935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:48.876848 23935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:48.876869 23935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:48.878281 23935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:48.878343 23935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:48.878357 23935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:48.878369 23935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:48.878381 23935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:48.880352 23935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:46365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:33599,127.1.132.189:41167,127.1.132.188:33741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:33741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:48.880596 23935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:48.880779 23935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:48.885634 23943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:48.885735 23941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:48.885634 23940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:48.886130 23935 server_base.cc:1034] running on GCE node
I20260923 22:16:48.886391 23935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:48.886683 23935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:48.887845 23935 hybrid_clock.cc:648] HybridClock initialized: now 1790201808887834 us; error 27 us; skew 500 ppm
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:48.890614 23935 init.cc:377] Logged in from keytab as kudu/127.1.132.188@KRBTEST.COM (short username kudu)
I20260923 22:16:48.891114 23935 webserver.cc:460] Webserver started at http://127.1.132.188:35289/ using document root <none> and password file <none>
I20260923 22:16:48.891328 23935 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:48.891376 23935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:48.891489 23935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:48.892639 23935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data/instance:
uuid: "1a2e5a0672a24be0a2cea55196c2c387"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "f937257aba11b3664407c0c248e9c4bc"
server_key_iv: "c0ce349cde47b38b25a4d86551baa538"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.892983 23935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal/instance:
uuid: "1a2e5a0672a24be0a2cea55196c2c387"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "f937257aba11b3664407c0c248e9c4bc"
server_key_iv: "c0ce349cde47b38b25a4d86551baa538"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.894311 23935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:48.895033 23950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:48.895174 23935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:16:48.895259 23935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "1a2e5a0672a24be0a2cea55196c2c387"
format_stamp: "Formatted at 2026-09-23 22:16:48 on dist-test-slave-l4q5"
server_key: "f937257aba11b3664407c0c248e9c4bc"
server_key_iv: "c0ce349cde47b38b25a4d86551baa538"
server_key_version: "encryptionkey@0"
I20260923 22:16:48.895342 23935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:48.907706 23928 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } attempt: 2
I20260923 22:16:48.908226 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.908057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41030)
0923 22:16:48.908122 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.908170 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:16:48.908567 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.908489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41040)
0923 22:16:48.908527 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.908551 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:16:48.908694 23928 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:33741: Network error: Client connection negotiation failed: client connection to 127.1.132.188:33741: connect: Connection refused (error 111)
I20260923 22:16:48.914625 23935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:48.916812 23935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:48.916949 23935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:48.920677 23935 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.188:33741
I20260923 22:16:48.920707 24002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.188:33741 every 8 connection(s)
I20260923 22:16:48.921037 23935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
I20260923 22:16:48.921999 24003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:48.923575 24003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.927029 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 23935
I20260923 22:16:48.927129 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201541420007-1554-0/minicluster-data/master-2/wal/instance
I20260923 22:16:48.927302 1554 external_mini_cluster.cc:1428] Setting key d31d0f50903b994c6e2deae862c3ee96
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:48.932942 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.924490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.1.132.190:33599)
0923 22:16:48.924557 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.924558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.924587 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:16:48.925072 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.925073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.925083 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.925134 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.925731 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.926289 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.927360 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.927603 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.927680 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.930052 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.930053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.930054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.930060 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.931451 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.931946 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.931946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.931947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.931994 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.932277 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.932278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.932278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.932407 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.932521 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.932768 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.932846 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:48.932996 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.924289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:33599 (local address 127.0.0.1:38132)
0923 22:16:48.924486 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.924524 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.924983 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.925207 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.925210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.925460 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.925652 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.925656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.926419 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.926420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.927279 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.927281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.927700 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.927702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.927859 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.928412 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.928423 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.929961 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.931530 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.931537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.931539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.931871 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.932054 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.932055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.932055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.932212 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.932510 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.932515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.932630 (+ 115us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.932707 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.932832 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
I20260923 22:16:48.933522 24003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.935966 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.927653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.1.132.188:33741)
0923 22:16:48.927797 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.927802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.927916 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:16:48.927982 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.927985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.928010 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.928077 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.928817 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.929504 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.930435 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.930880 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.930944 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.933453 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.933470 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.933470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.933477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.934900 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.935213 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.935216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.935216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.935265 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.935438 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.935439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.935439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.935573 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.935693 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.935780 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.935870 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:48.940689 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.933973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.1.132.189:41167)
0923 22:16:48.934032 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.934034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.934062 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:16:48.934132 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.934133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.934142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.934188 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.934473 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.934913 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.935557 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.935769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.935836 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.937930 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.937931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.937932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.937939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.939297 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.939798 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.939800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.939801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.939861 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.940145 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.940146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.940146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.940298 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.940363 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.940524 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.940594 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:16:48.940688 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.933811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:41167 (local address 127.0.0.1:54752)
0923 22:16:48.934019 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.934030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.934076 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.934215 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.934217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.934333 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.934394 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.934396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.934994 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.934995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.935479 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.935481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.935849 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.935849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.935919 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.936361 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.936366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.937792 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.939539 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.939540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.939540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.939703 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.939977 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.939978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.939978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.940040 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.940368 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.940369 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.940429 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.940493 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.940593 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:48.941162 24003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:48.943727 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.936687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.1.132.190:33599)
0923 22:16:48.936727 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.936729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.936736 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:48.936759 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.936760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.936770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.936830 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.937370 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.938009 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.938901 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.939096 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.939157 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.941306 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.941307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.941307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.941315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.942746 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.943028 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.943029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.943030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.943081 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.943247 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.943248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.943248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.943355 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.943505 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.943572 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.943641 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:48.947731 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.941502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.1.132.188:33741)
0923 22:16:48.941543 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.941544 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.941583 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:16:48.941651 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.941652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.941663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.941711 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.941929 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.942636 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.943296 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.943507 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.943561 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.945323 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.945324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.945325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.945332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.946658 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.946973 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.946973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.946974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.947019 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.947184 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.947185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.947185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.947309 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.947364 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.947511 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.947626 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":23}
I20260923 22:16:48.947734 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.941405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41060)
0923 22:16:48.941540 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.941550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.941596 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.941715 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.941716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.941833 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.941864 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.941865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.942727 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.942728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.943209 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.943210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.943568 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.943569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.943614 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.943944 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.943948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.945249 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.946730 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.946731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.946731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.946886 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.947078 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.947079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.947079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.947131 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.947374 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.947375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.947417 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.947490 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.947585 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260923 22:16:48.948922 23853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } attempt: 2
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:48.949623 24003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: Bootstrap starting.
I20260923 22:16:48.951063 24003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:48.951370 24003 log.cc:824] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:48.952212 24003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: No bootstrap required, opened a new log
Sep 23 22:16:48 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:48.952658 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.944537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.1.132.189:41167)
0923 22:16:48.944572 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.944574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.944584 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:48.944618 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.944619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.944628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.944663 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.945188 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.945822 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.946568 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.946837 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.946883 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.949489 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.949490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.949491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.949498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.950983 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.951363 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.951364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.951364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.951839 (+ 475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.952060 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.952061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.952061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.952214 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.952386 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.952498 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.952575 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:16:48.954095 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:48.954612 24003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.954769 24003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2e5a0672a24be0a2cea55196c2c387, State: Initialized, Role: FOLLOWER
I20260923 22:16:48.954847 24003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.955058 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.949159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41068)
0923 22:16:48.949219 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:48.949231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:48.949287 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:48.949399 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:48.949400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:48.949535 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:48.949573 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.949575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.950194 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.950195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.950678 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:48.950680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.951051 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.951052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:48.951108 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.951423 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:48.951428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:48.952654 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:48.954141 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.954142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.954143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.954300 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.954464 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:48.954465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:48.954465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:48.954521 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:48.954759 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:48.954760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:48.954804 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 22:16:48.954861 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:16:48.954970 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:16:48.955145 24003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:48.955099 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.955194 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:48.955350 23827 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ff202e85c18448588072837f236d2bcd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38148
I20260923 22:16:48.956423 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.949215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.1.132.188:33741)
0923 22:16:48.949246 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.949248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.949271 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:16:48.949331 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.949332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.949340 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.949393 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.949643 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.950120 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.950753 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.950964 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.951031 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.952737 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.952738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.952738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.952745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.954073 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.954364 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.954365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.954365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.954417 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.954576 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.954577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.954577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.954697 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.954751 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.956266 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.956347 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:16:48.956841 24020 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:48.956888 24020 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:48.957825 23853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: Bootstrap starting.
I20260923 22:16:48.958545 23853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:48.958835 23853 log.cc:824] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:48.959677 23853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: No bootstrap required, opened a new log
I20260923 22:16:48.957029 23935 master_runner.cc:186] Error getting master registration for 127.1.132.190:33599: OK, instance_id { permanent_uuid: "ff202e85c18448588072837f236d2bcd" instance_seqno: 1790201808605091 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:48.959767 23935 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:48.961066 23853 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.961217 23853 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff202e85c18448588072837f236d2bcd, State: Initialized, Role: FOLLOWER
I20260923 22:16:48.961297 23853 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.961621 23853 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:48.961580 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.961666 23827 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:16:48.961688 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:48.961720 23827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:48.962322 23827 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.962580 23827 leader_election.cc:288] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [CANDIDATE]: Term 1 election: Requested vote from peers 91b547a5954e459fb7fe421693faff2e (127.1.132.189:41167), 1a2e5a0672a24be0a2cea55196c2c387 (127.1.132.188:33741)
W20260923 22:16:48.964033 24034 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 22:16:48.963994 23977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff202e85c18448588072837f236d2bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1a2e5a0672a24be0a2cea55196c2c387"
W20260923 22:16:48.964097 24034 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:48.964119 23977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:48.965124 23977 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff202e85c18448588072837f236d2bcd in term 1.
I20260923 22:16:48.965544 23803 leader_election.cc:302] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2e5a0672a24be0a2cea55196c2c387, ff202e85c18448588072837f236d2bcd; no voters:
I20260923 22:16:48.965673 24023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:48.965955 24023 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [term 1 LEADER]: Becoming Leader. State: Replica: ff202e85c18448588072837f236d2bcd, State: Running, Role: LEADER
I20260923 22:16:48.966094 24023 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:48.966750 23785 master_runner.cc:186] Error getting master registration for 127.1.132.189:41167: OK, instance_id { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" instance_seqno: 1790201808761550 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:48.969161 23785 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:48.969434 24040 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff202e85c18448588072837f236d2bcd. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.969534 24040 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:48.969797 24042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:48.970155 24042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:48.971926 23977 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [term 1 FOLLOWER]: Refusing update from remote peer ff202e85c18448588072837f236d2bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:48.972309 24040 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:48.973740 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff202e85c18448588072837f236d2bcd. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.973851 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:48.980345 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.966215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.1.132.189:41167)
0923 22:16:48.966263 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.966265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.972811 (+ 6546us) server_negotiation.cc:407] Connection header received
0923 22:16:48.972955 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.972957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.972968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.973008 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.974127 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.974609 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.975625 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.975964 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.976020 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.977565 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.977566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.977567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.977594 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.979142 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.979509 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.979510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.979511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.979584 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.979779 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.979780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.979780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.979912 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.979990 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.980150 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.980247 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:48.987207 24040 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.987340 24040 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:48.987365 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:48.987450 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:48.987277 24047 mvcc.cc:205] Tried to move back new op lower bound from 7332666609546047488 to 7332666609526030336. Current Snapshot: MvccSnapshot[applied={T|T < 7332666609546047488}]
I20260923 22:16:48.990001 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.972572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.1.132.188:33741)
0923 22:16:48.972614 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.972616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.980644 (+ 8028us) server_negotiation.cc:407] Connection header received
0923 22:16:48.980739 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.980740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.980752 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.980779 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.981447 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.982128 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.983084 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.983356 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.983390 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.984834 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.984835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.984836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.984843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.988200 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.988791 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.988793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.988805 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.988867 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.989225 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.989226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.989227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.989386 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.989459 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.989812 (+ 353us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.989893 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:48.991492 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:48.972147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.1.132.190:33599)
0923 22:16:48.972225 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:48.972227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:48.980360 (+ 8133us) server_negotiation.cc:407] Connection header received
0923 22:16:48.980480 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:48.980481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:48.980494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:48.980524 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:48.981205 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.981939 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.982962 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:48.983244 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:48.983303 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:48.988346 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:48.988347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:48.988348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:48.988356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:48.990106 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.990697 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.990698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.990699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.990762 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:48.990948 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:48.990948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:48.990949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:48.991077 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:48.991167 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:48.991321 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:16:48.991398 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:48.996905 24042 catalog_manager.cc:1347] Generated new cluster ID: 16e2a8d7cabf4bd9aa421e999e85929a
I20260923 22:16:48.996989 24042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:48.999820 23928 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } attempt: 3
Sep 23 22:16:49 dist-test-slave-l4q5 krb5kdc[23772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:49.010636 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:49.000153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:33741 (local address 127.0.0.1:41074)
0923 22:16:49.000277 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:49.000290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:49.000337 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:49.000620 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:49.000621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:49.000804 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:49.000849 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:49.000851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:49.002341 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:49.002342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:49.003153 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:49.003156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:49.003766 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:49.003767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:49.003834 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:49.004399 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:49.004404 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:49.006270 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:49.009509 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:49.009511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:49.009511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:49.009742 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:49.009963 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:49.009963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:49.009964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:49.010028 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:49.010314 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:49.010315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:49.010383 (+ 68us) client_negotiation.cc:769] Sending connection context
0923 22:16:49.010434 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:16:49.010533 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260923 22:16:49.011206 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:49.000405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.1.132.188:33741)
0923 22:16:49.000443 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.000444 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.000451 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:49.000477 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.000477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.000487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.000535 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.000917 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:49.001590 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:49.003336 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:49.003648 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:49.003700 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:49.006387 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:49.006389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:49.006389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:49.006396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:49.009391 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:49.009824 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:49.009825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:49.009825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:49.009893 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:49.010089 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:49.010090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:49.010091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:49.010229 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:49.010278 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:49.011038 (+ 760us) server_negotiation.cc:299] Negotiation successful
0923 22:16:49.011121 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:16:49.012996 23928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: Bootstrap starting.
I20260923 22:16:49.013689 23928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:49.013942 23928 log.cc:824] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:49.021505 23928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: No bootstrap required, opened a new log
I20260923 22:16:49.023466 23928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:49.023627 23928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b547a5954e459fb7fe421693faff2e, State: Initialized, Role: FOLLOWER
I20260923 22:16:49.023694 23928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } }
I20260923 22:16:49.024036 23928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:49.024461 23901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:49.025350 23901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [term 1 FOLLOWER]: Refusing update from remote peer ff202e85c18448588072837f236d2bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:49.025769 24040 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:49.026091 23902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff202e85c18448588072837f236d2bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "91b547a5954e459fb7fe421693faff2e"
I20260923 22:16:49.026971 23902 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff202e85c18448588072837f236d2bcd in term 1.
I20260923 22:16:49.029513 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:49.030123 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:49.030220 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff202e85c18448588072837f236d2bcd. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:49.030308 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:49.030375 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:49.030449 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:49.036787 24054 mvcc.cc:205] Tried to move back new op lower bound from 7332666609546047488 to 7332666609526030336. Current Snapshot: MvccSnapshot[applied={T|T < 7332666609546047488}]
I20260923 22:16:49.038208 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff202e85c18448588072837f236d2bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff202e85c18448588072837f236d2bcd" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 33599 } } peers { permanent_uuid: "91b547a5954e459fb7fe421693faff2e" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 41167 } } peers { permanent_uuid: "1a2e5a0672a24be0a2cea55196c2c387" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 33741 } } }
I20260923 22:16:49.038328 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:49.038908 24067 catalog_manager.cc:1259] Loaded cluster ID: 16e2a8d7cabf4bd9aa421e999e85929a
I20260923 22:16:49.038962 24067 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: loading cluster ID for follower catalog manager: success
W20260923 22:16:49.039080 24067 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 22:16:49.039121 24067 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:49.039551 24042 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:49.040338 24042 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:49.052294 24042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff202e85c18448588072837f236d2bcd: Generated new TSK 0
I20260923 22:16:49.052557 24042 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:16:49.079587 23856 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.078192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.1.132.190:33599)
0923 22:16:49.078240 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.078242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.078882 (+ 640us) server_negotiation.cc:407] Connection header received
0923 22:16:49.078993 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.078994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.079006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.079035 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.079509 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38162: BlockingRecv error: recv got EOF from 127.0.0.1:38162 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 22:16:49.080850 24007 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.078561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.1.132.188:33741)
0923 22:16:49.078603 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.078604 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.080171 (+ 1567us) server_negotiation.cc:407] Connection header received
0923 22:16:49.080363 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.080365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.080377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.080406 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.080763 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41090: BlockingRecv error: recv got EOF from 127.0.0.1:41090 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:16:49.081671 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.079979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.1.132.189:41167)
0923 22:16:49.080020 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.080021 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.081074 (+ 1053us) server_negotiation.cc:407] Connection header received
0923 22:16:49.081174 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.081176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.081187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.081217 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.081604 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54776: BlockingRecv error: recv got EOF from 127.0.0.1:54776 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:16:49.084297 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.083425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.1.132.189:41167)
0923 22:16:49.083463 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.083465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.083674 (+ 209us) server_negotiation.cc:407] Connection header received
0923 22:16:49.083786 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.083787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.083807 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.083853 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.084245 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54788: BlockingRecv error: recv got EOF from 127.0.0.1:54788 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:16:49.087520 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.086679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54796 (local address 127.1.132.189:41167)
0923 22:16:49.086715 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.086717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.086971 (+ 254us) server_negotiation.cc:407] Connection header received
0923 22:16:49.087072 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.087074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.087086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.087115 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.087468 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54796: BlockingRecv error: recv got EOF from 127.0.0.1:54796 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:16:49.091692 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.090848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.1.132.189:41167)
0923 22:16:49.090888 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.090890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.091133 (+ 243us) server_negotiation.cc:407] Connection header received
0923 22:16:49.091235 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.091236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.091248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.091277 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.091637 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54810: BlockingRecv error: recv got EOF from 127.0.0.1:54810 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:49.098091 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.097157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.1.132.189:41167)
0923 22:16:49.097200 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.097201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.097454 (+ 253us) server_negotiation.cc:407] Connection header received
0923 22:16:49.097559 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.097561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.097573 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.097603 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.098034 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54814: BlockingRecv error: recv got EOF from 127.0.0.1:54814 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:16:49.108466 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.107526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.1.132.189:41167)
0923 22:16:49.107569 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.107571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.107832 (+ 261us) server_negotiation.cc:407] Connection header received
0923 22:16:49.107933 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.107935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.107946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.107974 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.108404 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54822: BlockingRecv error: recv got EOF from 127.0.0.1:54822 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:49.127151 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.126018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.1.132.189:41167)
0923 22:16:49.126062 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.126065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.126519 (+ 454us) server_negotiation.cc:407] Connection header received
0923 22:16:49.126639 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.126640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.126653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.126680 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.127089 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54830: BlockingRecv error: recv got EOF from 127.0.0.1:54830 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:16:49.161264 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.160548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.1.132.189:41167)
0923 22:16:49.160598 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.160600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.160755 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:16:49.160805 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.160806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.160818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.160868 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.161201 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54832: BlockingRecv error: recv got EOF from 127.0.0.1:54832 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:16:49.226890 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.226274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.1.132.189:41167)
0923 22:16:49.226359 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.226361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.226373 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:16:49.226432 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.226433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.226443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.226514 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.226841 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54838: BlockingRecv error: recv got EOF from 127.0.0.1:54838 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260923 22:16:49.356758 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.356058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.1.132.189:41167)
0923 22:16:49.356130 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.356133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.356170 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:16:49.356267 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.356269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.356281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.356332 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.356694 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54852: BlockingRecv error: recv got EOF from 127.0.0.1:54852 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260923 22:16:49.615063 23933 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:49.614151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.1.132.189:41167)
0923 22:16:49.614277 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:49.614279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:49.614286 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:49.614311 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:49.614312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:49.614321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:49.614448 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:49.615008 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54860: BlockingRecv error: recv got EOF from 127.0.0.1:54860 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260923 22:16:49.957310 24020 catalog_manager.cc:1259] Loaded cluster ID: 16e2a8d7cabf4bd9aa421e999e85929a
I20260923 22:16:49.957386 24020 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: loading cluster ID for follower catalog manager: success
I20260923 22:16:49.958210 24020 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: acquiring CA information for follower catalog manager: success
I20260923 22:16:49.958498 24020 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a2e5a0672a24be0a2cea55196c2c387: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:50.040385 24067 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: acquiring CA information for follower catalog manager: success
I20260923 22:16:50.040690 24067 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91b547a5954e459fb7fe421693faff2e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:50.131312 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:50.128457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.1.132.189:41167)
0923 22:16:50.128579 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:50.128582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:50.128647 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:16:50.128746 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:50.128747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:50.128855 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:50.128923 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:50.129147 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.129649 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.130478 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.130683 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.130745 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:50.130963 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:50.131076 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:50.131151 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:50.131204 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:16:50.131224 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260923 22:16:50.135735 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:50.133314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.1.132.188:33741)
0923 22:16:50.133439 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:50.133441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:50.133479 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:16:50.133548 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:50.133549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:50.133582 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:50.133651 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:50.133738 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.134212 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.135006 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.135205 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.135256 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:50.135444 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:50.135555 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:50.135609 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:50.135646 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:16:50.135666 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260923 22:16:50.140272 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:50.137502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.1.132.189:41167)
0923 22:16:50.137555 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:50.137557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:50.137655 (+ 98us) server_negotiation.cc:407] Connection header received
0923 22:16:50.137725 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:50.137726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:50.137735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:50.137780 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:50.137938 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.138672 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.139468 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.139731 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.139787 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:50.139985 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:50.140020 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:50.140046 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:50.140144 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:16:50.140168 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:50.140547 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:50.137759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.1.132.188:33741)
0923 22:16:50.137808 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:50.137810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:50.137922 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:16:50.138016 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:50.138017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:50.138027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:50.138071 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:50.138186 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.138869 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.139850 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.140118 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.140206 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:50.140329 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:50.140359 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:50.140410 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:50.140463 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 22:16:50.140484 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:16:50.140779 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:50.137479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.1.132.190:33599)
0923 22:16:50.137575 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:50.137577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:50.137625 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:16:50.137783 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:50.137785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:50.137855 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:50.137938 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:50.138100 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.138623 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.139666 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:50.139934 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:50.139987 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:50.140164 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:50.140312 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:50.140366 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:50.140668 (+ 302us) server_negotiation.cc:299] Negotiation successful
0923 22:16:50.140698 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":47,"threads_started":1}
I20260923 22:16:50.142787 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23785
I20260923 22:16:50.147574 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23860
I20260923 22:16:50.152098 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 23935
2026-09-23T22:16:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2754 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:16:50 dist-test-slave-l4q5 krb5kdc[24152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:16:50 dist-test-slave-l4q5 krb5kdc[24152](info): set up 2 sockets
Sep 23 22:16:50 dist-test-slave-l4q5 krb5kdc[24152](info): commencing operation
krb5kdc: starting...
W20260923 22:16:51.112239 1554 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/test-user.keytab.
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
Entry for principal kudu/127.1.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab.
I20260923 22:16:51.239483 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39821
--webserver_interface=127.1.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:16:51.319031 24165 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:51.319250 24165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:51.319268 24165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:51.319288 24165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:51.320497 24165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:51.320540 24165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:51.320554 24165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:51.320565 24165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:51.320575 24165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:51.322074 24165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.190.keytab
--principal=kudu/127.1.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.190:39821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:51.322311 24165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:51.322543 24165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:51.327227 24173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:51.327257 24171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:51.327281 24170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:51.327525 24165 server_base.cc:1034] running on GCE node
I20260923 22:16:51.327767 24165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:51.328055 24165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:51.329205 24165 hybrid_clock.cc:648] HybridClock initialized: now 1790201811329204 us; error 44 us; skew 500 ppm
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:51.331658 24165 init.cc:377] Logged in from keytab as kudu/127.1.132.190@KRBTEST.COM (short username kudu)
I20260923 22:16:51.332087 24165 webserver.cc:460] Webserver started at http://127.1.132.190:43231/ using document root <none> and password file <none>
I20260923 22:16:51.332314 24165 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:51.332365 24165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:51.332459 24165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:51.333441 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data/instance:
uuid: "606163d82b6f4d0387d7d1cf0123ce4a"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "ea9bd7557b0baaeefc90e858542854e2"
server_key_iv: "db1d03ed1241f1ddb2a8c5a98f9d10f3"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.333719 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance:
uuid: "606163d82b6f4d0387d7d1cf0123ce4a"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "ea9bd7557b0baaeefc90e858542854e2"
server_key_iv: "db1d03ed1241f1ddb2a8c5a98f9d10f3"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.334916 24165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:16:51.335613 24180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:51.335830 24165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:51.335947 24165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal
uuid: "606163d82b6f4d0387d7d1cf0123ce4a"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "ea9bd7557b0baaeefc90e858542854e2"
server_key_iv: "db1d03ed1241f1ddb2a8c5a98f9d10f3"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.336006 24165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:51.353675 24165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:51.355922 24165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:51.356043 24165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:51.360203 24165 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.190:39821
I20260923 22:16:51.360285 24232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.190:39821 every 8 connection(s)
I20260923 22:16:51.360599 24165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/data/info.pb
I20260923 22:16:51.361655 24233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:51.363265 24233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.364163 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 24165
I20260923 22:16:51.364291 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-0/wal/instance
I20260923 22:16:51.364509 1554 external_mini_cluster.cc:1428] Setting key c0b1fd7f512180c4d6bac2727e027ec8
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.189@KRBTEST.COM" created.
I20260923 22:16:51.374420 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.364486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39821 (local address 127.0.0.1:45650)
0923 22:16:51.364701 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.364741 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.365227 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.365562 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.365564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.365884 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.366053 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.366058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.367129 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.367130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.367834 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.367836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.368408 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.368409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.368472 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.369051 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.369059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.370728 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.372750 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.372757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.372759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.372966 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.373266 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.373267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.373268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.373384 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.373804 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.373809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.373903 (+ 94us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.373991 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.374237 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":96,"thread_start_us":98,"threads_started":1}
I20260923 22:16:51.374423 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.364509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.1.132.190:39821)
0923 22:16:51.364710 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.364723 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.364820 (+ 97us) server_negotiation.cc:407] Connection header received
0923 22:16:51.365379 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.365382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.365406 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.365519 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.366182 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.367031 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.367922 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.368311 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.368414 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.370849 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.370858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.370858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.370865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.372655 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.373059 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.373063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.373063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.373165 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.373464 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.373465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.373465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.373731 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.373807 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.374049 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.374161 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260923 22:16:51.375174 24233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.375522 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.375407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38356)
0923 22:16:51.375450 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.375502 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:16:51.375626 24182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
I20260923 22:16:51.376258 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.376145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38362)
0923 22:16:51.376202 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.376234 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:16:51.376363 24233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:35603: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
Entry for principal kudu/127.1.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab.
I20260923 22:16:51.391594 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:35603
--webserver_interface=127.1.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:51.425622 24233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } attempt: 1
I20260923 22:16:51.426153 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.426022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38372)
0923 22:16:51.426088 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.426121 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 22:16:51.426589 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.426457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38382)
0923 22:16:51.426531 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.426564 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260923 22:16:51.426681 24233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:35603: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
W20260923 22:16:51.475769 24240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:51.476017 24240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:51.476034 24240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:51.476058 24240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:51.477322 24240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:51.477376 24240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:51.477389 24240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:51.477401 24240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:51.477415 24240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:51.478996 24240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.189.keytab
--principal=kudu/127.1.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.189:35603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:51.479249 24240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:51.479476 24240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:51.484208 24246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:51.484323 24248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:51.484437 24240 server_base.cc:1034] running on GCE node
W20260923 22:16:51.484450 24245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:51.484717 24240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:51.485003 24240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:51.486143 24240 hybrid_clock.cc:648] HybridClock initialized: now 1790201811486127 us; error 30 us; skew 500 ppm
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:51.488631 24240 init.cc:377] Logged in from keytab as kudu/127.1.132.189@KRBTEST.COM (short username kudu)
I20260923 22:16:51.489086 24240 webserver.cc:460] Webserver started at http://127.1.132.189:33015/ using document root <none> and password file <none>
I20260923 22:16:51.489281 24240 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:51.489322 24240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:51.489413 24240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:51.490404 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data/instance:
uuid: "8ea647632f734e3d99e0c36594e869c5"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "7779c227666965ed6b7133c8ae6a6e0a"
server_key_iv: "77618417e77233fd4632475eeb03ddd7"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.490689 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal/instance:
uuid: "8ea647632f734e3d99e0c36594e869c5"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "7779c227666965ed6b7133c8ae6a6e0a"
server_key_iv: "77618417e77233fd4632475eeb03ddd7"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.491950 24240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 22:16:51.492731 24255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:51.492965 24240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:16:51.493042 24240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal
uuid: "8ea647632f734e3d99e0c36594e869c5"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "7779c227666965ed6b7133c8ae6a6e0a"
server_key_iv: "77618417e77233fd4632475eeb03ddd7"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.493103 24240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:51.494843 24233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } attempt: 2
I20260923 22:16:51.495401 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.495167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38392)
0923 22:16:51.495314 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.495361 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260923 22:16:51.495834 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.495726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38396)
0923 22:16:51.495773 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.495808 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:16:51.496027 24233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.189:35603: Network error: Client connection negotiation failed: client connection to 127.1.132.189:35603: connect: Connection refused (error 111)
I20260923 22:16:51.513561 24240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:51.515677 24240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:51.515821 24240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:51.519834 24240 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.189:35603
I20260923 22:16:51.519870 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.189:35603 every 8 connection(s)
I20260923 22:16:51.520246 24240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/data/info.pb
I20260923 22:16:51.521258 24308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:51.522840 24308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.526808 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 24240
I20260923 22:16:51.526909 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-1/wal/instance
I20260923 22:16:51.527076 1554 external_mini_cluster.cc:1428] Setting key 5d53e80d4c434fc7415b19e284404420
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
I20260923 22:16:51.532658 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.523613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.1.132.190:39821)
0923 22:16:51.523689 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.523691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.523914 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:16:51.524448 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.524449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.524463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.524517 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.525111 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.525646 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.526629 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.526858 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.526935 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.529415 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.529417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.529417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.529426 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.531004 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.531486 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.531488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.531489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.531561 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.531881 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.531882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.531882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.532041 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.532117 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.532465 (+ 348us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.532559 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:16:51.532754 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.523555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39821 (local address 127.0.0.1:45658)
0923 22:16:51.523772 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.523820 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.524351 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.524581 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.524584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.524833 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.525030 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.525034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.525718 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.525718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.526528 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.526531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.527007 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.527009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.527129 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.527714 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.527723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.529288 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.531110 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.531119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.531122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.531389 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.531657 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.531658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.531659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.531803 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.532131 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.532138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.532324 (+ 186us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.532438 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.532574 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260923 22:16:51.533347 24308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.132.188@KRBTEST.COM" created.
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:51.541376 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.533631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38408)
0923 22:16:51.533684 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.533696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.533741 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.533957 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.533958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.534101 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.534170 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.534172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.535044 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.535045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.535575 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.535577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.536233 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.536234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.536323 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.536682 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.536687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.538627 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.540357 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.540358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.540358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.540563 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.540741 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.540742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.540742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.540801 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.541044 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.541045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.541096 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.541163 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.541276 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:16:51.541488 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.533682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.1.132.189:35603)
0923 22:16:51.533826 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.533830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.533845 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:16:51.533863 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.533864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.533875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.533935 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.534236 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.534966 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.535666 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.536105 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.536171 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.538716 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.538729 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.538729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.538737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.540271 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.540639 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.540642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.540642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.540693 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.540849 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.540850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.540850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.540991 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.541047 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.541196 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.541359 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260923 22:16:51.542164 24308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.542567 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.542425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36520)
0923 22:16:51.542480 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.542537 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 22:16:51.542709 24256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
I20260923 22:16:51.543134 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.543032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36526)
0923 22:16:51.543083 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.543113 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 22:16:51.543244 24308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:41031: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
Entry for principal kudu/127.1.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab.
I20260923 22:16:51.555770 1554 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1S3mcj/build/release/bin/kudu
/tmp/dist-test-task1S3mcj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:41031
--webserver_interface=127.1.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.132.148:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:16:51.587225 24233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } attempt: 3
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:51.592482 24308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } attempt: 1
I20260923 22:16:51.592936 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.592784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36538)
0923 22:16:51.592865 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.592896 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 22:16:51.593317 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.593206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36552)
0923 22:16:51.593256 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.593286 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 22:16:51.593403 24308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:41031: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
I20260923 22:16:51.595053 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.587645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38418)
0923 22:16:51.587707 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.587720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.587766 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.587928 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.587929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.588088 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.588150 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.588152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.588854 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.588855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.589402 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.589404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.589868 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.589870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.589949 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.590375 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.590379 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.591861 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.593755 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.593756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.593756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.593973 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.594257 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.594258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.594258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.594320 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.594719 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.594720 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.594774 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.594833 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.594957 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:16:51.595049 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.587730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.1.132.189:35603)
0923 22:16:51.587807 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.587810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.587819 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:51.587852 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.587853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.587867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.587910 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.588250 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.588757 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.589495 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.589756 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.589858 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.591970 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.591972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.591973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.591982 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.593660 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.594127 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.594128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.594129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.594196 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.594477 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.594478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.594479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.594614 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.594697 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.594871 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.594959 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:16:51.595564 24233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.595976 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.595829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36560)
0923 22:16:51.595873 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.595944 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 22:16:51.596287 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.596164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36574)
0923 22:16:51.596239 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.596264 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260923 22:16:51.596375 24233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:41031: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
I20260923 22:16:51.627610 24233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } attempt: 1
I20260923 22:16:51.628216 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.628013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36588)
0923 22:16:51.628083 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.628141 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 22:16:51.628624 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.628510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36602)
0923 22:16:51.628554 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.628597 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:16:51.628762 24233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:41031: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
W20260923 22:16:51.640383 24315 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:16:51.640614 24315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:16:51.640631 24315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:16:51.640653 24315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:16:51.641922 24315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:16:51.641978 24315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:16:51.641991 24315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:16:51.642002 24315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:16:51.642014 24315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:16:51.643635 24315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.132.148:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.132.190:39821,127.1.132.189:35603,127.1.132.188:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/krb5kdc/kudu_127.1.132.188.keytab
--principal=kudu/127.1.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.132.188:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 21:02:32 UTC on ccycloud.quasar-lecndr.root.comops.site
build id 4668598
I20260923 22:16:51.643889 24315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:16:51.644125 24315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:16:51.649044 24323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:16:51.649046 24320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:51.649108 24315 server_base.cc:1034] running on GCE node
W20260923 22:16:51.649044 24321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:16:51.649540 24315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:16:51.649873 24315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:16:51.651041 24315 hybrid_clock.cc:648] HybridClock initialized: now 1790201811651013 us; error 44 us; skew 500 ppm
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:16:51.654318 24315 init.cc:377] Logged in from keytab as kudu/127.1.132.188@KRBTEST.COM (short username kudu)
I20260923 22:16:51.655078 24315 webserver.cc:460] Webserver started at http://127.1.132.188:43935/ using document root <none> and password file <none>
I20260923 22:16:51.655472 24315 fs_manager.cc:359] Metadata directory not provided
I20260923 22:16:51.655550 24315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:16:51.655691 24315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:16:51.657261 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data/instance:
uuid: "6bf35901462049f69f7e93345540b0cd"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "9175a807801ec2304165fe8e439ac006"
server_key_iv: "d12546936bd9290fbe7484ebe1160b79"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.657601 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal/instance:
uuid: "6bf35901462049f69f7e93345540b0cd"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "9175a807801ec2304165fe8e439ac006"
server_key_iv: "d12546936bd9290fbe7484ebe1160b79"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.659279 24315 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260923 22:16:51.660017 24330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:16:51.660233 24315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:16:51.660326 24315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data,/tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal
uuid: "6bf35901462049f69f7e93345540b0cd"
format_stamp: "Formatted at 2026-09-23 22:16:51 on dist-test-slave-l4q5"
server_key: "9175a807801ec2304165fe8e439ac006"
server_key_iv: "d12546936bd9290fbe7484ebe1160b79"
server_key_version: "encryptionkey@0"
I20260923 22:16:51.660388 24315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:16:51.661538 24308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } attempt: 2
I20260923 22:16:51.661943 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.661823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36612)
0923 22:16:51.661877 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.661911 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:16:51.662243 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.662162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36628)
0923 22:16:51.662199 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.662222 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:16:51.662322 24308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.132.188:41031: Network error: Client connection negotiation failed: client connection to 127.1.132.188:41031: connect: Connection refused (error 111)
I20260923 22:16:51.680873 24315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:16:51.683027 24315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:16:51.683177 24315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:16:51.687050 24315 rpc_server.cc:307] RPC server started. Bound to: 127.1.132.188:41031
I20260923 22:16:51.687103 24382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.132.188:41031 every 8 connection(s)
I20260923 22:16:51.687433 24315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/data/info.pb
I20260923 22:16:51.688457 24383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:16:51.690004 24383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.691004 1554 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1S3mcj/build/release/bin/kudu as pid 24315
I20260923 22:16:51.691097 1554 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1S3mcj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201541420007-1554-0/minicluster-data/master-2/wal/instance
I20260923 22:16:51.691268 1554 external_mini_cluster.cc:1428] Setting key bb5f822daa34e81a6b4fd4a469b0ea2c
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:51.700713 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.690736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.1.132.190:39821)
0923 22:16:51.690862 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.690864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.690998 (+ 134us) server_negotiation.cc:407] Connection header received
0923 22:16:51.691687 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.691688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.691712 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.691761 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.692655 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.693130 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.694179 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.694463 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.694528 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.697282 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.697284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.697284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.697294 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.699047 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.699551 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.699552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.699553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.699614 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.699917 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.699919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.699919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.700074 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.700139 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.700523 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.700614 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260923 22:16:51.700891 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.690669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.190:39821 (local address 127.0.0.1:45660)
0923 22:16:51.690863 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.690914 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.691494 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.691883 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.691888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.692300 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.692543 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.692550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.693219 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.693220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.694094 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.694096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.694579 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.694580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.694715 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.695402 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.695411 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.697128 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.699153 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.699160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.699162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.699471 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.699679 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.699680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.699681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.699815 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.700165 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.700173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.700380 (+ 207us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.700479 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.700634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260923 22:16:51.701426 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.691618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.1.132.188:41031)
0923 22:16:51.691758 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.691763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.691784 (+ 21us) server_negotiation.cc:407] Connection header received
0923 22:16:51.691874 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.691876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.691892 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.691962 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.692682 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.693661 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.694522 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.694965 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.695026 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.697649 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.697658 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.697659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.697666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.699504 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.699996 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.699999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.700000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.700055 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.700351 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.700352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.700352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.700499 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.700646 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.700735 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.701326 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260923 22:16:51.701628 24383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.703905 24233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } attempt: 2
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.190@KRBTEST.COM
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.190@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:51.708643 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.702128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.1.132.189:35603)
0923 22:16:51.702181 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.702183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.702192 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:51.702262 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.702263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.702274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.702320 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.702564 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.703022 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.703638 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.703842 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.703896 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.705864 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.705865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.705866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.705873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.707339 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.707796 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.707798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.707798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.707859 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.708074 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.708074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.708075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.708246 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.708304 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.708471 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.708551 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:16:51.708966 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.702099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.189:35603 (local address 127.0.0.1:38430)
0923 22:16:51.702136 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.702148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.702186 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.702332 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.702333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.702457 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.702498 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.702500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.703088 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.703089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.703566 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.703568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.703901 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.703902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.703966 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.704408 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.704412 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.705766 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.707435 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.707436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.707437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.707688 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.707943 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.707944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.707944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.708006 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.708299 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.708300 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.708343 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.708772 (+ 429us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.708865 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:16:51.709378 24383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:16:51.709475 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.702212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.1.132.190:39821)
0923 22:16:51.702249 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.702250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.702256 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:16:51.702323 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.702324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.702333 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.702371 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.702804 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.703248 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.704214 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.704473 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.704531 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.706623 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.706624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.706625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.706631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.708128 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.708562 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.708563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.708563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.708615 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.708825 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.708827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.708827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.708953 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.709087 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.709304 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.709385 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:16:51.711184 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.704103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36646)
0923 22:16:51.704141 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.704150 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.704210 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.704347 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.704348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.704497 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.704536 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.704538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.705369 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.705370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.706103 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.706106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.706763 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.706765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.706822 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.707251 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.707256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.708641 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.710256 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.710257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.710257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.710442 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.710625 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.710625 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.710626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.710682 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.710899 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.710900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.710943 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.711003 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.711094 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:16:51.711181 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.704222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.1.132.188:41031)
0923 22:16:51.704264 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.704266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.704272 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:16:51.704289 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.704290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.704299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.704343 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.704615 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.705275 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.706181 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.706476 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.706545 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.708720 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.708721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.708721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.708728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.710201 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.710511 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.710512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.710512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.710568 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.710738 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.710739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.710740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.710839 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.710893 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.711034 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.711095 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:51.713385 24233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: Bootstrap starting.
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.188@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:51.713991 24233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:51.714295 24233 log.cc:824] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: Log is configured to *not* fsync() on all Append() calls
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.132.189@KRBTEST.COM
I20260923 22:16:51.715260 24233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: No bootstrap required, opened a new log
I20260923 22:16:51.716956 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.709844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36652)
0923 22:16:51.710013 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.710022 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.710056 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.710219 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.710219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.710341 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.710386 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.710388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.711355 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.711355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.711995 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.711997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.712432 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.712433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.712490 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.712819 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.712824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.714131 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.715866 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.715867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.715867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.716044 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.716284 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.716284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.716285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.716336 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.716625 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.716625 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.716657 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.716723 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.716822 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260923 22:16:51.717051 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.709908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.1.132.188:41031)
0923 22:16:51.710064 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.710066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.710076 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:51.710097 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.710098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.710108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.710175 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.710551 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.711296 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.712069 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.712351 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.712414 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.714220 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.714221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.714222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.714228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.715774 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.716111 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.716112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.716112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.716159 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.716388 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.716389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.716389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.716550 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.716605 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.716765 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.716879 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260923 22:16:51.717366 24233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.717487 24233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606163d82b6f4d0387d7d1cf0123ce4a, State: Initialized, Role: FOLLOWER
I20260923 22:16:51.717562 24233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.717911 24233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:51.717947 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.718029 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.718109 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.710471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.1.132.189:35603)
0923 22:16:51.710508 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.710510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.710517 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:16:51.710560 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.710560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.710569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.710617 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.711195 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.711749 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.712589 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.712880 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.712947 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.715210 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.715212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.715212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.715219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.716548 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.716880 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.716881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.716881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.716944 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.717564 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.717565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.717566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.717685 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.717829 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.717941 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.718022 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:51.718354 24383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: Bootstrap starting.
I20260923 22:16:51.719054 24383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:51.719338 24383 log.cc:824] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: Log is configured to *not* fsync() on all Append() calls
W20260923 22:16:51.719713 24401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:51.719803 24401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:51.719897 1554 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:16:51.720417 24383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: No bootstrap required, opened a new log
I20260923 22:16:51.720813 24207 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "606163d82b6f4d0387d7d1cf0123ce4a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45664
I20260923 22:16:51.720912 24207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:16:51.720942 24207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:51.721544 24207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.721724 24207 leader_election.cc:288] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [CANDIDATE]: Term 1 election: Requested vote from peers 8ea647632f734e3d99e0c36594e869c5 (127.1.132.189:35603), 6bf35901462049f69f7e93345540b0cd (127.1.132.188:41031)
I20260923 22:16:51.720381 24165 master_runner.cc:186] Error getting master registration for 127.1.132.189:35603: OK, instance_id { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" instance_seqno: 1790201811518705 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:51.722744 24165 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:51.724987 24383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.725143 24383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf35901462049f69f7e93345540b0cd, State: Initialized, Role: FOLLOWER
I20260923 22:16:51.725224 24383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.725535 24383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:51.725493 24411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.725570 24357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6bf35901462049f69f7e93345540b0cd"
I20260923 22:16:51.725621 24411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.725682 24357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:51.726564 24357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606163d82b6f4d0387d7d1cf0123ce4a in term 1.
I20260923 22:16:51.726827 24181 leader_election.cc:302] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606163d82b6f4d0387d7d1cf0123ce4a, 6bf35901462049f69f7e93345540b0cd; no voters:
I20260923 22:16:51.726974 24390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:16:51.727114 24390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [term 1 LEADER]: Becoming Leader. State: Replica: 606163d82b6f4d0387d7d1cf0123ce4a, State: Running, Role: LEADER
I20260923 22:16:51.727196 24390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
W20260923 22:16:51.727360 24422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:16:51.727414 24422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:51.727654 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606163d82b6f4d0387d7d1cf0123ce4a. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.727748 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:51.728114 24426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:16:51.728322 24426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:16:51.729231 24315 master_runner.cc:186] Error getting master registration for 127.1.132.189:35603: OK, instance_id { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" instance_seqno: 1790201811518705 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:16:51.731839 24315 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:16:51.739642 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.723156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.1.132.188:41031)
0923 22:16:51.723227 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.723241 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.723382 (+ 141us) server_negotiation.cc:407] Connection header received
0923 22:16:51.723516 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.723517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.723538 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.723565 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.729229 (+ 5664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.729998 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.733236 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.733574 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.733648 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.736341 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.736343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.736343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.736351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.737909 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.738492 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.738493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.738494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.738565 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.738873 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.738873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.738874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.739025 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.739122 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.739451 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.739544 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:16:51.741616 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.722940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.1.132.189:35603)
0923 22:16:51.722980 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.722982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.723123 (+ 141us) server_negotiation.cc:407] Connection header received
0923 22:16:51.723370 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.723371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.723394 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.723421 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.728328 (+ 4907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.729078 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.734482 (+ 5404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.734808 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.734851 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.737065 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.737067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.737067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.737075 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.739246 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.740457 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.740458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.740458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.740526 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.740879 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.740881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.740881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.741024 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.741103 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.741433 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.741518 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:16:51.742089 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.723113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.1.132.190:39821)
0923 22:16:51.723153 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.723155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.723197 (+ 42us) server_negotiation.cc:407] Connection header received
0923 22:16:51.723325 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.723326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.723336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.723377 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.731175 (+ 7798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.732229 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.734161 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.734462 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.734525 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.737351 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.737352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.737352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.737360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.739946 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.740746 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.740747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.740747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.740825 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.741289 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.741290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.741290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.741439 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.741543 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.741919 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.741997 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:51.743990 24357 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [term 1 FOLLOWER]: Refusing update from remote peer 606163d82b6f4d0387d7d1cf0123ce4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:51.744403 24423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:51.745906 24411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606163d82b6f4d0387d7d1cf0123ce4a. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.746030 24411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.747318 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.747416 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: This master's current role is: LEADER
I20260923 22:16:51.747440 24411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.747520 24411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.748062 24426 catalog_manager.cc:1347] Generated new cluster ID: 52fb181af4ac44c9b9a87c628f33c58a
I20260923 22:16:51.748114 24426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:16:51.753417 24308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } attempt: 3
Sep 23 22:16:51 dist-test-slave-l4q5 krb5kdc[24152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201811, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.132.189@KRBTEST.COM for kudu/127.1.132.188@KRBTEST.COM
I20260923 22:16:51.760758 24426 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:16:51.762181 24426 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:16:51.762588 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.753798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.1.132.188:41031)
0923 22:16:51.753837 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.753839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.753850 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:16:51.753935 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.753936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.753946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.753992 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.754361 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.755093 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.756120 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.756453 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.756527 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.759691 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:16:51.759692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:16:51.759692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:16:51.759701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:16:51.761372 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.761783 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.761784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.761785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.761847 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:16:51.762029 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:16:51.762030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:16:51.762031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:16:51.762174 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:16:51.762235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:51.762398 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:16:51.762489 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:16:51.762588 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:51.753747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.132.188:41031 (local address 127.0.0.1:36670)
0923 22:16:51.753807 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:16:51.753819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:16:51.753872 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:16:51.754041 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:16:51.754042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:16:51.754228 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:16:51.754281 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.754284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.755177 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.755179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.756035 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:16:51.756038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:16:51.756527 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:16:51.756528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:16:51.756582 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:51.757307 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:16:51.757321 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:16:51.759581 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:16:51.761462 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.761463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.761464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.761697 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.761911 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:16:51.761912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:16:51.761912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:16:51.761979 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:16:51.762240 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:16:51.762241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:16:51.762299 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 22:16:51.762374 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:16:51.762489 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 22:16:51.764282 24308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: Bootstrap starting.
I20260923 22:16:51.765080 24308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: Neither blocks nor log segments found. Creating new log.
I20260923 22:16:51.765410 24308 log.cc:824] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:16:51.769212 24308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: No bootstrap required, opened a new log
I20260923 22:16:51.771423 24308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.771587 24308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea647632f734e3d99e0c36594e869c5, State: Initialized, Role: FOLLOWER
I20260923 22:16:51.771667 24308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } }
I20260923 22:16:51.772020 24308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:16:51.772233 24282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8ea647632f734e3d99e0c36594e869c5"
I20260923 22:16:51.772337 24282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:16:51.773243 24282 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606163d82b6f4d0387d7d1cf0123ce4a in term 1.
I20260923 22:16:51.773470 24281 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [term 1 FOLLOWER]: Refusing update from remote peer 606163d82b6f4d0387d7d1cf0123ce4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:16:51.774050 24431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:16:51.777519 24426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a: Generated new TSK 0
W20260923 22:16:51.777706 24450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 22:16:51.777782 24426 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:16:51.777797 24450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:16:51.777995 24439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.778147 24439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.778234 24439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606163d82b6f4d0387d7d1cf0123ce4a. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.778337 24439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.778402 24439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.778488 24439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:16:51.784530 24437 mvcc.cc:205] Tried to move back new op lower bound from 7332666620956270592 to 7332666620835516416. Current Snapshot: MvccSnapshot[applied={T|T < 7332666620900966400}]
I20260923 22:16:51.784999 24423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606163d82b6f4d0387d7d1cf0123ce4a" member_type: VOTER last_known_addr { host: "127.1.132.190" port: 39821 } } peers { permanent_uuid: "8ea647632f734e3d99e0c36594e869c5" member_type: VOTER last_known_addr { host: "127.1.132.189" port: 35603 } } peers { permanent_uuid: "6bf35901462049f69f7e93345540b0cd" member_type: VOTER last_known_addr { host: "127.1.132.188" port: 41031 } } }
I20260923 22:16:51.785106 24423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606163d82b6f4d0387d7d1cf0123ce4a [sys.catalog]: This master's current role is: LEADER
W20260923 22:16:51.795827 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.794013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.1.132.189:35603)
0923 22:16:51.794056 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.794058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.794990 (+ 932us) server_negotiation.cc:407] Connection header received
0923 22:16:51.795092 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.795093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.795104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.795134 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.795700 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38448: BlockingRecv error: recv got EOF from 127.0.0.1:38448 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 22:16:51.796662 24236 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.794402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.1.132.190:39821)
0923 22:16:51.794438 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.794450 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.796111 (+ 1661us) server_negotiation.cc:407] Connection header received
0923 22:16:51.796238 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.796239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.796250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.796277 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.796598 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45682: BlockingRecv error: recv got EOF from 127.0.0.1:45682 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:16:51.797444 24388 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.794735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.1.132.188:41031)
0923 22:16:51.794772 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.794774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.796903 (+ 2129us) server_negotiation.cc:407] Connection header received
0923 22:16:51.797013 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.797014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.797026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.797054 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.797382 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36684: BlockingRecv error: recv got EOF from 127.0.0.1:36684 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:51.800995 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.799131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.1.132.189:35603)
0923 22:16:51.799171 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.799173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.799404 (+ 231us) server_negotiation.cc:407] Connection header received
0923 22:16:51.799508 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.799509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.799529 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.799557 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.800938 (+ 1381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38450: BlockingRecv error: recv got EOF from 127.0.0.1:38450 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:16:51.804281 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.803402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.1.132.189:35603)
0923 22:16:51.803439 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.803441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.803678 (+ 237us) server_negotiation.cc:407] Connection header received
0923 22:16:51.803786 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.803787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.803799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.803827 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.804167 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38462: BlockingRecv error: recv got EOF from 127.0.0.1:38462 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:16:51.808513 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.807592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.1.132.189:35603)
0923 22:16:51.807634 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.807636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.807876 (+ 240us) server_negotiation.cc:407] Connection header received
0923 22:16:51.807972 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.807973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.807986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.808013 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.808455 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38472: BlockingRecv error: recv got EOF from 127.0.0.1:38472 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:51.814793 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.813854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.1.132.189:35603)
0923 22:16:51.813899 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.813901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.814171 (+ 270us) server_negotiation.cc:407] Connection header received
0923 22:16:51.814294 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.814295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.814308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.814338 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.814734 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38478: BlockingRecv error: recv got EOF from 127.0.0.1:38478 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 22:16:51.825253 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.824259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.1.132.189:35603)
0923 22:16:51.824318 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.824321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.824627 (+ 306us) server_negotiation.cc:407] Connection header received
0923 22:16:51.824737 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.824739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.824752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.824782 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.825189 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38492: BlockingRecv error: recv got EOF from 127.0.0.1:38492 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260923 22:16:51.843650 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.842699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.1.132.189:35603)
0923 22:16:51.842745 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.842748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.843025 (+ 277us) server_negotiation.cc:407] Connection header received
0923 22:16:51.843129 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.843130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.843143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.843173 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.843589 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38498: BlockingRecv error: recv got EOF from 127.0.0.1:38498 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 22:16:51.878037 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.877122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.1.132.189:35603)
0923 22:16:51.877168 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.877171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.877452 (+ 281us) server_negotiation.cc:407] Connection header received
0923 22:16:51.877548 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.877549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.877561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.877590 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.877974 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38502: BlockingRecv error: recv got EOF from 127.0.0.1:38502 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 22:16:51.944124 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:51.943452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.1.132.189:35603)
0923 22:16:51.943501 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:51.943504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:51.943620 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:16:51.943683 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:51.943684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:51.943696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:51.943744 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:51.944068 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38514: BlockingRecv error: recv got EOF from 127.0.0.1:38514 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 22:16:52.073969 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:52.073338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.1.132.189:35603)
0923 22:16:52.073383 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.073386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.073451 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:16:52.073519 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.073520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.073532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.073579 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:52.073907 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38528: BlockingRecv error: recv got EOF from 127.0.0.1:38528 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:16:52.332070 24312 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:16:52.331342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.1.132.189:35603)
0923 22:16:52.331444 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.331446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.331456 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:16:52.331482 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.331483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.331493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.331556 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:16:52.332006 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38542: BlockingRecv error: recv got EOF from 127.0.0.1:38542 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 22:16:52.727875 24422 catalog_manager.cc:1259] Loaded cluster ID: 52fb181af4ac44c9b9a87c628f33c58a
I20260923 22:16:52.727944 24422 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: loading cluster ID for follower catalog manager: success
I20260923 22:16:52.728801 24422 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: acquiring CA information for follower catalog manager: success
I20260923 22:16:52.729092 24422 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bf35901462049f69f7e93345540b0cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:52.778291 24450 catalog_manager.cc:1259] Loaded cluster ID: 52fb181af4ac44c9b9a87c628f33c58a
I20260923 22:16:52.778368 24450 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: loading cluster ID for follower catalog manager: success
I20260923 22:16:52.779258 24450 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: acquiring CA information for follower catalog manager: success
I20260923 22:16:52.779551 24450 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ea647632f734e3d99e0c36594e869c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:16:52.847980 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:52.845392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.1.132.189:35603)
0923 22:16:52.845530 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.845533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.845542 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:52.845571 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.845572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.845624 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.845692 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:52.845857 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.846361 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.847193 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.847394 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.847450 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:52.847624 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:52.847740 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:52.847816 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:52.847868 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:16:52.847891 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260923 22:16:52.853240 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:52.849930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.1.132.188:41031)
0923 22:16:52.850021 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.850023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.850173 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:16:52.850248 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.850249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.850303 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.850394 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:52.850503 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.851244 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.852262 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.852579 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.852650 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:52.852820 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:52.852976 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:52.853054 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:52.853108 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 22:16:52.853136 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"thread_start_us":45,"threads_started":1}
I20260923 22:16:52.858418 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:52.855271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.1.132.188:41031)
0923 22:16:52.855316 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.855318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.855480 (+ 162us) server_negotiation.cc:407] Connection header received
0923 22:16:52.855541 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.855542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.855553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.855600 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:52.855780 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.856554 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.857658 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.857946 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.858000 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:52.858174 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:52.858208 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:52.858254 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:52.858308 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 22:16:52.858333 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:16:52.858497 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:52.855232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.1.132.189:35603)
0923 22:16:52.855353 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.855355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.855754 (+ 399us) server_negotiation.cc:407] Connection header received
0923 22:16:52.855784 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.855785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.855797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.855839 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:52.855973 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.856712 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.857678 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.857907 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.857962 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:52.858111 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:52.858148 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:52.858188 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:52.858406 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 22:16:52.858433 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260923 22:16:52.858747 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:16:52.855123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.1.132.190:39821)
0923 22:16:52.855231 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:16:52.855233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:16:52.855242 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:16:52.855289 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:16:52.855290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:16:52.855355 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:16:52.855442 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:16:52.855642 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.856430 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.857520 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:16:52.857826 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:16:52.858174 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:16:52.858258 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:16:52.858381 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:16:52.858626 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0923 22:16:52.858643 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 22:16:52.858666 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
I20260923 22:16:52.865382 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 24165
I20260923 22:16:52.871445 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 24240
I20260923 22:16:52.875982 1554 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1S3mcj/build/release/bin/kudu with pid 24315
2026-09-23T22:16:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2723 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5478 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271461 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST